8

I have been working in iOS project. Regarding Xcode Derived Data folder whose contents are generated by build system, there are many Stackoverflow posts, i.e. how to remove derived data folder or is it safe to remove derived data folder, etc. but there is none which I am looking for. My question is to know when should I delete the derived data folder and when should I not? In other word, in which conditions this derived data becomes non reusable and why? Are those derived data become stale after resource or configuration change of the project or anything else? I often see many issues are resolved after deleting derived data but we often do not try to know the root cause of the issues.

Can I simply say, until the .xcodeproj file change, the derived data can be reused safely?

Background of the question: In my project, I have a proof build system so that every CLs need to pass the proof build before getting merged to remote branch. So I need to know the risks to architect when I should (or should not) reuse the derived data folder to make the proof build system performant.

Sazzad Hissain Khan
  • 37,929
  • 33
  • 189
  • 256

4 Answers4

3

You might want to clean derived data when -

  1. When you pull code from remote repository because it might contain new/deleted file which you might be using in other module/project.

Consider there are two project (say ProjectA and ProjectB) in your workspace.

1) You already build your app and their derived data is available.
3) Now consider there is reference to some `Class` of Project A in Project B. You have used static function form  ProjectA classes.
2) Now you pull code from remote repo and it contains changes for Project A only.
3) In this pull, the used static function form ProjectA gets deleted.
3) When you build app again, only Project A gets complied again but not Project B because it has no code change. Inshort, when there is change in code XCode complies that module again not complete app.
4) Now you will you get build issue as dependencies are not correct now. Project B has no idea what happened.
5) So in this case, you should clean derived data.
  1. Correct code coverage report.

Code coverage reports into the default derived data directory located at ~/Library/Developer/Xcode/DerivedData. You can clean code coverge report form DerivedData and regerate it again.

Please correct me if i am wrong. This is as per my understanding.

Will update this answer if i found more reasons.

DevesH
  • 486
  • 4
  • 18
3

Sadly, deleting DerivedData has become the catch all to try and remedy build issues. As you already know, you can delete the DerivedData at any point in time with no issues (unless of course you were building). The nuances as to when it is safe is a little more complicated. In practice I have generally found that significant changes in the code between builds can have issues. Likewise, changing Xcode versions can also have issues. The key words here being can, which does not mean it will.

I do not believe there is any definitive means of knowing when to purge, which is why I think you find most people generally asking "did you remove DerivedData" as one of the first questions when someone has a build problem.

IMHO, I think there is some subtle caching that Xcode does within DerivedData, as I've had build failures in weird situations I could not trace. I would not be surprised if Xcode has some subtle bug related to this caching. In general, I only trusted a command line delete of the project's DerivedData. And if I was extra careful, I'd also purge any associated libraries/frameworks as well as the ModuleCache (in DerivedData). There is also some voodoo in the /var/folders. Not sure if this is still the case, but I have found myself surgically nuking stuff there. Yeah, bad. But I'm also the kind of person that has been known to edit the project.pbxproj file to fix it, so I guess I throw caution to the wind.

Let me ask you this question. What is more important? "bullet-proof" or build times? I've been roped into maintaining or upgrading the build systems for a few apps. I've always done everything as a clean build. This always was fresh pull from source control and then having a defined DerivedData within the workspace (these were Jenkins builds and was configured to use a clean workspace). This pretty much avoided build issues related to needing to remove DerivedData. I've seen way too many strange issues occur due to some Xcode voodoo. It is far better to not add red herrings if you have a build error.

If you do find that the builds as well as any tests needed are taking too long, you can start to devise other strategies to help make it more manageable. This becomes very project dependent. For example on a project I was working on, I moved all libs to pre-built versions. For some reason, they had it setup where they had to build all the 3rd party libs (like openssl) during the build process. This also means that the devs incurred this build penalty the first time they built or any time the cleaned.

Mobile Ben
  • 7,121
  • 1
  • 27
  • 43
  • off-course bulletproof is the primary goal but if I can be sure at which point I can re use the cache to reduce the build time there is no harm to apply it. Surely, to get the advantage I need to know the exact points. My project is a large one and takes more than 20 mins to build and test and so, I am trying to apply as much options as possible to make it performant without sacrificing its sanity. – Sazzad Hissain Khan Nov 28 '19 at 04:52
  • I agree with you with your goals. Unfortunately from what I've experienced, Xcode as too many build issues. I've seen things like files which were modified not being built. Hence places I've been at, we've taken a more conservative approach. This meant getting faster and more build machines in some cases. Others it was revisiting what we're building and either doing pre-builts or even splitting up the project. For one project, we ended up breaking it up into 3 different projects (2 frameworks and the main project). – Mobile Ben Nov 28 '19 at 05:06
  • Something you could try perhaps would be to reuse `DerivedData`, but then have a more "more thorough" build that happens at fixed intervals during the day (like once a day or more). – Mobile Ben Nov 28 '19 at 05:08
1

Derived data can take up significant space too. Derived data is generated during each app build process. Derived data contains intermediate build results, generated indexes, that help speed up build time. You can think of derived data as cached data/image in web browser, most of the time web browser will auto-save the data/image into your hard disk when you first browse a web page, and the next time you visit the same web page, the page load speed is faster as web browser will use the cached data /image in hard disk.

Few cases I would like to mention when you should delete derived data:

  • I faced this issue many times that Xcode’s debugger stop working suddenly so resolve this issue I delete derived data and it start working fine.
  • Whenever you are facing issues in building project then it is better to delete derived data and build again.
  • Deleting derived data causes minor delay in building project for the first time but it deletes all old settings for the project.
  • Deleting derived data boost up your system’s performance as it holds lots of memory space which may not needed since very long.
  • And last but not least for error that are not logical sometimes, all you have to do is just delete Derived data & error is gone.

Happy to help!

Shiv Jaiswal
  • 539
  • 1
  • 6
  • 22
  • That is not the answer of my question. I already know many issues are resolved after deleting derived data. My question is when these derived data become stale so that using this cache will hamper the expected behavior of my app. – Sazzad Hissain Khan Nov 26 '19 at 12:14
0

When can i delete the derived data ?

If you happen to have multiple build agents running on multiple machines (either physical or virtual), then each of those agents should do it’s own cleanup.Cleaning derived data might increase the time of first build for each project next day, but it’s a minor drawback. You will also claim free space back by killing DerivedData’s huge appetite. A practical advice to take home - clean Xcode Derived Data on regular basis on your CI box(es),You could create a cron job to do that, make it run some time after midnight and execute this simple shell command.

rm -rf /Users/username/Library/Developer/Xcode/DerivedData/*

For daily use on your development machine create a type alias in your bash profile.

typealias xcode-clean-derived="rm -rf /Users/i4niac/Library/Developer/Xcode/DerivedData/*"

Source : https://mgrebenets.github.io/mobile%20ci/2015/02/01/xcode-derived-data

redhatvicky
  • 1,912
  • 9
  • 8
  • What if I use single agent? Do I still need to clean for each build? If so, what purpose the derived data serve? – Sazzad Hissain Khan Dec 02 '19 at 06:34
  • Xcode uses derived data to make builds faster but includes index, build output and logs inside this directory. Overtime this could take up significant space on your hard drive. You can always Rebuild your dependencies so its good to free up the space , when its a single agent then it depends on your build schedule which can take either a long time/short to make this folder bigger/smaller. – redhatvicky Dec 02 '19 at 06:40
  • Let's say I have infinite amount of storage, the question is specific about the point to which the derived data becomes stale. question has not been answered in your post. – Sazzad Hissain Khan Dec 02 '19 at 06:45
  • AFAIK , There are tons of possibilities when your derived data becomes stale , Random crashes, slowness, autocomplete not working for a few seconds and build errors right after you’ve added or removed a library. Or, just a random appearance of 200+ warnings.. The solution for quite a few problems with Xcode’s building process can be found in clearing out your derived data. So i see this as one possibility. – redhatvicky Dec 02 '19 at 06:50
  • Are you guessing these possibilities or you have proper reference? – Sazzad Hissain Khan Dec 02 '19 at 06:56