As for why I'd want to do this, see this question: What are the downsides to using the official Windows Git binaries from cygwin?
Or you could look at this: http://git.661346.n2.nabble.com/Fwd-Error-with-git-svn-pushing-a-rename-td7599382.html It prevents you from pushing renamed files to the svn server.
Cygwin is stuck at a version of git that has numerous bugs when used with the git-svn bridge. A lot of these have been fixed in the latest git (their master branch, not the one available for download). So how do you get, build and use it?
I'll answer this below.