I have found googling the name of the DMG and looking for .edu sites is a good way. EDU domain sites are generally schools or universities, and have better download speeds for me at least. For xcode 3.2.6, for example, University of California in San Fransisco has the DMG here: http://www.msg.ucsf.edu/local/programs/MacOSX/
I am commenting a while after this was asked because the other answer, while valid, still gives half the download speed that this site does when I test both (not simultaneously) with axel
or downthemall.
With axel
I can even abort a download at one server if I think I can find better, switch to another server, and enter the same command but with a different URL to the file, and it will resume where it left off, even on a completely different server!
This is the command I ended up using:
http://www.msg.ucsf.edu/local/programs/MacOSX/xcode_3.2.6_and_ios_sdk_4.3.dmg
But be aware that there are risks - the usual ones, like someone having tampered with the file. Make sure the files are the right size and if you notice anything fishy, look up the DMG's checksum. Generally people have posted their checksums online in troubleshooting threads and such.