Yeah, I had this problem - this is a duplicate of this:
XCode 6 GM: Constantly freezing / locking while editing Swift code
I filed bug #18316444 and their response was:
Engineering has provided the following information:
We believe this issue has been addressed in the latest Xcode 6.1 beta.
This pre-release version of the Xcode 6.1 developer tools is required
when developing for OS X Yosemite, and includes the Swift programming
language. Xcode 6.1 beta runs on OS X Mavericks and OS X Yosemite
Developer Preview.
Please test with this release, and update your bug report with the
results.
Xcode 6.1 beta - Build 6A1042b
https://developer.apple.com/xcode/downloads/
Please let us know if that resolves the issue for you by updating your
bug report.
I don't know if this will help or frustrate you, but the problem eventually went away for me. What I did end up doing though was having an open terminal ready will killall primed for SourceKit. I wish I knew what changed, it just did. One thing to check though, on early versions of XCode 6, I kept running out of pty terminals - it would leak, run out, and then SourceKit would "jam up". I had to up it to 512, I think (whatever the default was, I ended up doubling twice). I can't remember if that also worked around this issue, or if it was unaffected.
If you remain afflicted, you might consider editing with 6.1 and doing final builds on 6.0.x.