Although you can come close in C# usually the C++ version is faster (in my experience). The C++ optimizing compilers do a better job (can take longer to compile/optimize) and also in C++ you can use stuff like SSE to speed math stuff up, which is very cumbersome from C#. I have created code for large integer multiplication that is 5x faster in SSE as in assembler, which was only 10% faster as the same C++ code. The 5x came mostly from SSE being able to do 2x 32bit multiplies with a single instruction.
C# allows for unsafe code where you can do some pointer stuff. Speeding up things like array processing (as it normally bounds checks all array access this helps for array intense math processing). And you can fall back to using unmanaged DLL's written in C++
So unless your code will run for hours at an end I think optimizing it will probably be fun, but not really worth it ;-)
There was/is a C++/CLI variant for .NET from microsoft, allowing for easier mixing of C++ and C#. See http://www.functionx.com/cppcli/Lesson01.htm But not sure it that is still supported and from what I gathered it was frowned upon from the start, not sure why though.
You might also find this stack exchange question interesting about why VirtualMachine Languages differ in performance compared to precompiled ones; Why are JIT-ed languages still slower and less memory efficient than native C/C++?