Update for Xcode 13.4.1:
I had this infamous bug today as well. I tried a lot including clean project, delete derived, restart max etc.
What did the fix finally is similar to the answer of kakaiikaka: I set the workspace settings to "Legacy Build System (Deprecated)" for both, the shared and per user workspace. I tried to build with this, but got an error because I had packages wich are not supported.
Restarted Xcode, then changed back the build system. Restarted Xcode again.
Now the crash doesn't happen any more.
Looks like something internal was spoilt and cycling the build system fixed it.