31

We need to test our app in the context of an iOS upgrade (e.g., 5.1 -> 6.0). Unfortunately, Apple doesn't allow downgrading devices. We thought of doing it in the simulator, but different versions of the simulator are different environments in themselves. I think we can copy the bundle from one simulator to the other, but that won't migrate the keychain (will it?).

Thanks!

Vladimir Gritsenko
  • 1,669
  • 11
  • 25
  • 1
    What do you want to test by this? Testing the app separately on 5.1 and on 6.0 should be enough. The application can't run when the system is upgrading. – Sulthan Oct 03 '12 at 10:56
  • 3
    There actually are ways to downgrade your OS - have a look for tiny umbrella - e.g. http://www.idownloadblog.com/2012/05/25/save-your-shsh-blobs-5-1-1/ – Till Oct 03 '12 at 11:54
  • 1
    Sulthan, ah, the different between theory and practice... we think there's some logic in our app that breaks in the transition and leaves some of the app data in an inconsistent state. Till, thanks, we'll try that, if we find any device that we haven't upgraded to iOS 6.0 yet. Still, it would be easiest through the simulator. – Vladimir Gritsenko Oct 03 '12 at 12:27
  • Are you storing anything with CoreData? – mginn Apr 10 '15 at 17:00
  • 1
    Of course one could ask why you're bothering? If you're targeting the app store nobody is running iOS 5 or 6. Maybe more clarity as to why you think you need to test the upgrade. And to answer the question - there is essentially no great way of testing this. – Cameron Lowell Palmer Apr 10 '15 at 17:01
  • Apple does make this very difficult. We generally use unit tests that fake the version number. I think the only time we use the os version this way is to send a "firstTimeThisOSVersion" analytics call every time the OS version changes so we know how many unique users have upgraded to the latest system. – EricS Apr 13 '15 at 21:41
  • check this answer http://stackoverflow.com/questions/12468180/simulating-app-upgrade-in-ios – Mustafa Ibrahim Apr 13 '15 at 23:14
  • @MustafaIbrahim that question is about upgrading an *app*, whereas this one is about upgrading *iOS version* while staying on the same app version – CupawnTae Apr 14 '15 at 08:54
  • You want to test this because apps that were built for iOS N can behave differently on iOS N+1 than identical app (same code) built for iOS N+1. For example requesting location permissions process is different in iOS 11 for newly built apps and built for iOS 10. – DanSkeel Aug 25 '17 at 13:30

2 Answers2

11

To test a transition from one state (before) to another (after), you need a way to put the app in the before state.

Your app surely won't be running while the OS is being updated, so you really only need to worry about the app starting up and discovering that the OS has been updated. There are a couple options:

  • Copy all your app's data files from a device running the "old" iOS version (5.1 according to your question) to a device running the new (6.0) version. The organizer in Xcode will let you easily copy your app's "container" from a device to your Mac or vice versa.

  • Make your app write it's data in the "old" format. It's not uncommon for an app to have methods for reading and writing data in different formats depending on the environment, so it's often easier to get your app to write data out in the old format than to actually copy from an old device.

Whichever path you choose, think about any other places (like user defaults) where you might made OS version-dependent changes and set those back to values that correspond to the previous OS. This applies especially to keychain items, which aren't stored in your app's sandbox.

Unit testing frameworks (like Apple's XCTest framework) generally have a setup mechanism that you could use to reset your app to the before state, including copying files, adding and removing keychain items, setting defaults items, etc. You can then add unit tests that run whatever code might be involved in an update and test the results. With a set of easily repeatable tests you'll be able to debug any problems more easily.

However you approach it, the goal is to put the app in the same state that it would be in if it were running for the first time after an OS update occurred. You don't have to worry about simulating the actual OS update, you only need to trick the app into thinking that the update has just happened.

Caleb
  • 124,013
  • 19
  • 183
  • 272
  • 1
    Does this cover keychain entries (main point of the question I thought)? – CupawnTae Apr 17 '15 at 06:48
  • @CupawnTae The *put the app in the before state* idea certainly applies to keychain items, but since the keychain is (the) one thing that persists outside of the app's sandbox, you'd need to write code to do that. Thanks for pointing it out -- I'll edit to address keychain specifically. – Caleb Apr 17 '15 at 12:04
5

For now, you can still install iOS 8.2. When a new version is released, Apple leaves both versions open for installation for a short time. While that "signing window" is open you can upgrade a device, test, and then restore it from an image of the older version. So you could do some intensive testing while the window is open, but obviously that's not a long-term solution (it typically lasts only a few days).

If you have the budget for it, you could install 8.2 on a device, put a big sticker on it saying "do not upgrade", and keep it on 8.2 for as long as it's relevant. Install your app on that device and take a backup (with backup encryption enabled so that keychain entries will be included), then restore that backup to another device that's on 8.3 - this is basically the same procedure you'll go through when doing an upgrade/restore through iTunes so it should be pretty close. It won't be exactly the same as an OTA update on-device of course, but for that, see option 1 above (and see it soon).

CupawnTae
  • 14,192
  • 3
  • 29
  • 60