Apologies for the difficulties with VS 2017 and CUDA 9 RC.
Microsoft released VS 2017 Update 3 (15.3) on 8/14/2017, right after CUDA 9 RC was published. Unfortunately, this update results in an incompatibility with CUDA 9 RC. NVIDIA expects that the CUDA 9 GA (future) release will address this particular incompatibility. In the meantime, if you switch to using VS 2017 RTM (the very first release of VS 2017) with no updates, it should work with CUDA 9 RC. I'm not suggesting this is easy or difficult (in fact it may be impossible, unless you've previously archived an offline installer to do that), or providing exact steps to get VS 2017 (original) RTM here.
In other respects, the supported environments should be spelled out in the windows installation guide that ships with CUDA 9 RC, and which is also linked from the CUDA 9 RC download page on developer.nvidia.com. Based on this, the other option seems to be to switch to VS 2015 (still available), or else a VS 2015 toolchain within VS 2017.