There is always some miniscule chance of duplicates, but Globally Unique Identifiers are meant to be just that: globally unique ...not system-wide unique, but as in the Planet Earth unique.
I speculate that, theoretically, you have a better chance of duplicating a UUID across multiple systems, than on a single system. While the OS isn't going to store each GUID that it generates, it probably uses some time based seed data to avoid collisions in itself. Of course this is dependent on the implementation.
Oh, and chances...well there are 3.4 x 10^38 available, Wikipedia says you're more likely to get hit by a meteorite.
I'll also offer an alternative method, the Path.GetTempFileName() method may be worth looking into, because it has protection against collision...though it can only create 65,535 unique file names before throwing an exception if previous files aren't deleted.
Other than that, it's not very difficult to do:
string path;
do
{
path = Guid.NewGuid().ToString(); // Format as needed
} while (File.Exists(path));