2

There are number of possible solutions to do file backup application. I need to know which method would be rock-solid and professional way to perform copying of data files even though the file is being used or very large sized.

There is a known method called Volume shadow copy (VSS), however I've read that it is an overkill for a simple copying operation and instead the PInvoke BackupRead can be used.

.NET framework provides it's own methods:
File.Copy was (and possibly still is) problematic with large files and sharing the resources
FileStream seems to be suitable for backup purposes however I didn't locate comprehensive description and I am not sure if I'm correct.

Could you please enlighten me which method should be used (maybe I have overlooked some options) and why? If the VSS or PInvoke methods are preferred could you please also provide an example how to use it or some reference to comprehensive documentation (particularly I'm interested in the correct settings to create file handle, which would allow sharing the resources when the file is in use).

Thanks in advance.

Community
  • 1
  • 1
Almis
  • 171
  • 1
  • 2
  • 11

1 Answers1

2

Everything you're going to try in a live (i.e. currently running OS) volume will suffer from not being able to open some files. The reason is, applications and the OS itself opens files exclusively - that is, they open the files with ShareMode=0. You won't be able to read those files.

VSS negotiates with VSS-aware applications to release their open files for the duration, but relatively few applications outside Microsoft are VSS aware.

An alternative approach is to boot to another OS (on a USB stick or another on-disk volume) and do your work from there. For example, you could use the Microsoft Preinstallation environment (WinPE). You can, with some effort run a .Net 4.x application from there. From such an environment, you can get to pretty much any file on the target volume without sharing violations.

WinPE runs as local administrator. As such, you need to assert privileges, such as SE_BACKUP_NAME, SE_RESTORE_NAME, SE_SECURITY_NAME, SE_TAKE_OWNERSHIP_NAME, and you need to open the files with a flag called BACKUP_SEMANTCS...as described here.

The BackupRead/BackupWrite APIs are effective, if awkward. You can't use asynchronous file handles with these APIs...or at least MS claims you're in for "subtle errors" if you do. If those APIs are overkill, you can just use FileStreams.

There are bunches of little gotchas either way. For example, you should know when there are hardlinks in play or you'll be backing up redundant data...and when you restore, you don't want to break those links. There are APIs for getting all the hard links for a given file...NtQueryInformationFile, for example.

ReparsePoints (Junctions and SymLinks) require special handling, too...as they are low-level redirects to other file locations. You can run in circles following these reparse points if you're not careful, and even find yourself inadvertently backing up off-volume data.

Not easy to deal with all this stuff, but if thoroughness is an issue, you'll encounter them all before you're done.

Clay
  • 4,999
  • 1
  • 28
  • 45
  • Thanks for an answer. Doesn't the OS handle the issues with file ShareMode? The thing is that you do not want to run something like WinPE if you need to set the backup on a daily basis (i.e. every night). Volume shadow copy seems to work fine as far as I've tested. Am I wrong? – Almis Oct 20 '16 at 05:33
  • It is fine. I was just contrasting BackupRead/Write with FileCopy and identifying the structural issues present on the volume. VSS is fine under most circumstances. We use win pe because we have more stringent forensic requirement in our backups (I work on an enterprise backup/restore) – Clay Oct 20 '16 at 13:10