First: check KVO basics.
Second: in objective-C you would the removing observers code into try-catch block and live happy. The apple's guide tell the same:
Asking to be removed as an observer if not already registered as one
results in an NSRangeException. You either call
removeObserver:forKeyPath:context: exactly once for the corresponding
call to addObserver:forKeyPath:options:context:, or if that is not
feasible in your app, place the removeObserver:forKeyPath:context:
call inside a try/catch block to process the potential exception.
In swift there’s no KVO API call you can make to ask, “Is X observing key path Y of object Z?” There is some ways to workaround it.
Also check the one of the reasons of the crashes when removing the observer. Here is the quote:
"It" refers to the observer. -removeObserver:forKeyPath: raises this
exception if told to remove an object that isn't currently registered
as an observer. So what's happening is that a table view is trying to
unregister as an observer from one of your objects that it
unfortunately didn't previouly register as an observer for.
The usual cause of this is that you have a property that isn't KVO-
compliant. Something accesses your 'foo' property and registers as an
observer of that property, and also as an observer of the object
that's the property's current value; you change the value of 'foo'
without letting anyone know; the observer then later decides to stop
observing, gets your 'foo' property, and removes itself as an observer
of that object. But it's no longer the same object that it registered
as an observer for...