I would not be so sure that C will be faster than C# for what you are going to do. The jitted code is extremely fast. Also, if you are doing lots of calculations, by using c#/.net, you could potentially take advantage of the Task Parallel Library, which would greatly ease parallelizing your calculations. And really, with all of the multi-core machines, I think that you will get a lot more bang for your bucks if you can take advantage of multiple cores (of course, straight C can thread as well, but the TPL in .net 4 is much more productive than the base WinApi)
One other thing that you might consider, and I am speaking from outside of my own experience, so definitely do your own research: I have heard that F# can be a very good choice for writing scientific sets/calculation libraries. I haven't written a line of F# myself, but from what I understand, the language design supports writing the calculations in a way that makes them extremely parallelizable.
(of course, now you have a new problem -- learning F# ;)
there is a post here that you might check out: F# performance in scientific computing