Yes, you can only block specific list, you don't have permission to invert the case. This is against Apple's policy. Not sure if possible by using private apis, otherwise by public/legal way, you cant do that.
This is text from apple's documentation:
When a phone receives an incoming call, the system first consults the user’s block list to determine whether a call should be blocked. If the phone number is not on a user- or system-defined block list, the system then consults your app’s Call Directory extension to find a matching blocked number.
Above text clearly says that, when call received, it only consult your list to check for block list, invert case is not available.
Even you can remove blocked contacts added by other apps, you have only access to blocked numbers added by your app's call directory extension.
So, If you are thinking of blocking all numbers except some list, then this is not possible in iOS.