9

I'm trying to move a file using SetFileInformationByHandle. This technique has been proposed by Niall Douglas in his CppCon2015 talk "Racing The File System" as a way to atomically move/rename a file. However, I'm struggling to provide correct arguments; it always fails and GetLastError returns ERROR_INVALID_PARAMETER.

I've tried this with the following setups, using the Unicode Character Set:

  • VS2015U1, running the exe under Windows 10
  • VS2015U2, running the exe under Windows Server 2012
  • VS2013, running the exe under Windows 7

But the behaviour is the same. I made sure to have access to the test folders and test file.

#include <sdkddkver.h>
#include <windows.h>

#include <cstring>
#include <iostream>
#include <memory>

int main()
{
    auto const& filepath = L"C:\\remove_tests\\file.txt";
    auto const& destpath = L"C:\\remove_tests\\other.txt";
    // unclear if that's the "root directory"
    auto const& rootdir = L"C:\\remove_tests";

    // handles will be leaked but that should be irrelevant here
    auto const f_handle = CreateFile(filepath,
        GENERIC_READ | GENERIC_WRITE | DELETE,
        0,
        NULL,
        CREATE_ALWAYS,
        FILE_ATTRIBUTE_NORMAL,
        NULL);

    if (f_handle == INVALID_HANDLE_VALUE)
    {
        auto const err = GetLastError();
        std::cerr << "failed to create test file: " << err;
        return err;
    }

    auto const parent_dir_handle = CreateFile(rootdir,
        GENERIC_READ | GENERIC_WRITE,
        FILE_SHARE_READ | FILE_SHARE_WRITE,
        NULL,
        OPEN_EXISTING,
        FILE_ATTRIBUTE_NORMAL | FILE_FLAG_BACKUP_SEMANTICS,
        NULL);

    if (parent_dir_handle == INVALID_HANDLE_VALUE)
    {
        auto const err = GetLastError();
        std::cerr << "failed to get handle to parent directory: " << err;
        return err;
    }

    auto const destpath_bytes_with_null = sizeof(destpath);
    // unclear if we need to subtract the one wchar_t of FileNameLength:
    auto const struct_size = sizeof(FILE_RENAME_INFO) + destpath_bytes_with_null;
    auto const buf = std::make_unique<char[]>(struct_size);

    auto const fri = reinterpret_cast<FILE_RENAME_INFO*>(buf.get());
    fri->ReplaceIfExists =  TRUE; // as described by Niall Douglas
    fri->RootDirectory = parent_dir_handle;
    // with or without null terminator?
    fri->FileNameLength = destpath_bytes_with_null;
    std::memcpy(fri->FileName, destpath, destpath_bytes_with_null);

    BOOL res = SetFileInformationByHandle(f_handle, FileRenameInfo,
                                          fri, struct_size);
    if (!res)
    {
        auto const err = GetLastError();
        std::cerr << "failed to rename file: " << err;
        return err;
    }
    else
        std::cout << "success";
}

In particular, my questions are:

  • What is the "root directory" as required by FILE_RENAME_INFO?
  • Which permissions are required for the handles?
  • What's the underlying problem of the ERROR_INVALID_PARAMETER produced by SetFileInformationByHandle?
dyp
  • 38,334
  • 13
  • 112
  • 177
  • Same problem as [this one](http://stackoverflow.com/questions/36217150/deleting-a-file-based-on-disk-id). – Hans Passant Apr 06 '16 at 12:49
  • @HansPassant Well I had no trouble using `SetFileInformationByHandle` to *delete* files. However I'm having trouble using it to *move* files. Also, I'm not using `OpenFileById`. Could you please be a bit more specific as to what the similarity is between the issue I have and the questions/answers you've linked to? – dyp Apr 06 '16 at 13:15
  • 1
    It behaves the exact same way, remove DELETE to see that. I'm guessing it is a Win10 specific issue, somebody ought to talk to MSFT about it. I'll volunteer you. – Hans Passant Apr 06 '16 at 13:20
  • @HansPassant I've tried it now under Windows 7 and Windows Server 2012 as well, and the behaviour is the same as on Windows 10. Unfortunately, I still fail to see the similarity to the linked post on deleting files by disk id. When I remove the `DELETE` flag, I get an access denied error, whereas the problem in the linked question is (as far as I understand it) that `OpenFileByDiskId` seems to ignore the `DELETE` flag, such that the resulting handle is not fit to use with `SetFileInformationByHandle` to *delete* the file. – dyp Apr 07 '16 at 07:32
  • @Gerardo I'll first have to test it on several operating systems. Then I'll upvote. Is there any need to hurry accepting your answer? My plan was to wait with accepting an answer and giving the bounty until the end of the bounty period. – dyp Apr 09 '16 at 09:44

2 Answers2

5

The documentation for SetFileInformationByHandle with FileRenameInfo and FILE_RENAME_INFO contains some errors. FILE_RENAME_INFO.FileNameLength must be set to the number of characters copied to FILE_RENAME_INFO.FileName excluding the terminating zero, and FILE_RENAME_INFO.RootDirectory must be null, even if moving the file from one directory to another.

#include <sdkddkver.h>
#include <windows.h>

#include <cstring>
#include <iostream>
#include <memory>

int _tmain( int argc, _TCHAR* argv [] )
{
    wchar_t* filename = L"C:\\remove_tests\\file.txt";
    wchar_t* destFilename = L"C:\\remove_tests2\\other.txt";

    // handles will be leaked but that should be irrelevant here
    auto fileHandle = CreateFile( filename,
                                  GENERIC_READ | GENERIC_WRITE | DELETE,
                                FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
                                      NULL,
                                      OPEN_EXISTING,
                                      FILE_ATTRIBUTE_NORMAL,
                                      NULL );

    if ( fileHandle == INVALID_HANDLE_VALUE )
    {
        auto const err = GetLastError( );
        std::cerr << "failed to create test file: " << err;
        return err;
    }

    auto destFilenameLength = wcslen( destFilename );

    auto bufferSize = sizeof( FILE_RENAME_INFO ) + ( destFilenameLength*sizeof( wchar_t ));
    auto buffer = _alloca( bufferSize );
    memset( buffer, 0, bufferSize );

    auto const fri = reinterpret_cast<FILE_RENAME_INFO*>( buffer );
    fri->ReplaceIfExists = TRUE;

    fri->FileNameLength = destFilenameLength;
    wmemcpy( fri->FileName, destFilename, destFilenameLength );

    BOOL res = SetFileInformationByHandle( fileHandle, FileRenameInfo, fri, bufferSize );
    if ( !res )
    {
        auto const err = GetLastError( );
        std::cerr << "failed to rename file: " << err;
        return err;
    }
    else
        std::cout << "success";
}
Unheilig
  • 16,196
  • 193
  • 68
  • 98
  • Thanks for your answer. How did you determine that the documentation contains errors? By experiment, or do you have another source? – dyp Apr 09 '16 at 09:42
  • [At another site](http://www.codeproject.com/Members/Espen-Harlinn) I have a slightly different ranking - and I think you'll find that I have some experience related to the ins and outs of the Windows API. It is rather obvious that something isn’t quite right with regards to the documentation for using SetFileInformationByHandle and FILE_RENAME_INFO. The FileName member of FILE_RENAME_INFO is Unicode based, so it was kind of natural to try out the number of Unicode characters. – Espen Harlinn Apr 09 '16 at 22:09
  • Well from my tests, it seems that the `FileNameLength` member is ignored entirely, the `RootDirectory` handle seems to be the culprit. I was mainly wondering because I'd appreciate learning about alternative sources of information about the Windows API, MSDN sometimes not being enough (as it is here). This being your first answer on StackOverflow, and one hour after Gerardo's answer (containing essentially the same hints) also made me a bit suspicious, I have to admit :) – dyp Apr 09 '16 at 22:41
4

I change a couple of thinks:

1) i dont use root handle (i set it to NULL)

2) i change your FILE_RENAME_INFO memory allocation code

NOTE: checked in windows 8, moving file in the same volume (disk)

auto const& filepath = L"C:\\remove_tests\\file.txt";
auto const& destpath = L"C:\\remove_tests\\other.txt";
// unclear if that's the "root directory"
auto const& rootdir = L"C:\\remove_tests";

// handles will be leaked but that should be irrelevant here
auto const f_handle = CreateFile(filepath,
    GENERIC_READ | GENERIC_WRITE | DELETE,
      0,
    NULL,
    CREATE_ALWAYS,
    FILE_ATTRIBUTE_NORMAL,
    NULL);

if (f_handle == INVALID_HANDLE_VALUE)
{
    auto const err = GetLastError();
    std::cerr << "failed to create test file: " << err;
    return err;
}

/*auto const parent_dir_handle = CreateFile(rootdir,
    GENERIC_READ | GENERIC_WRITE | DELETE,
      FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
    NULL,
    OPEN_EXISTING,
    FILE_ATTRIBUTE_NORMAL | FILE_FLAG_BACKUP_SEMANTICS,
    NULL);

if (parent_dir_handle == INVALID_HANDLE_VALUE)
{
    auto const err = GetLastError();
    std::cerr << "failed to get handle to parent directory: " << err;
    return err;
}*/

 auto const destpath_bytes_withOUT_null = _tcslen(destpath);
// unclear if we need to subtract the one wchar_t of FileNameLength:
auto const struct_size = sizeof(FILE_RENAME_INFO) + (destpath_bytes_withOUT_null + 1) * sizeof(WCHAR);
FILE_RENAME_INFO* fri = (FILE_RENAME_INFO*)new BYTE[struct_size];

fri->ReplaceIfExists =  TRUE; // as described by Niall Douglas
fri->RootDirectory = NULL;//parent_dir_handle;
// with or without null terminator?
fri->FileNameLength = destpath_bytes_withOUT_null;// No include null
 _tcscpy_s(fri->FileName, destpath_bytes_withOUT_null + 1, destpath);

BOOL res = SetFileInformationByHandle(f_handle, FileRenameInfo,
                                      fri, struct_size);

 delete fri;
if (!res)
{
    auto const err = GetLastError();
    std::cerr << "failed to rename file: " << err;
    return err;
}
else
    std::cout << "success";
Ing. Gerardo Sánchez
  • 1,607
  • 15
  • 14
  • Thanks for your answer. I've checked it now under Windows 10 and it works; tests on other operating systems will have to wait until Monday. I guess **not** using the root handle would be the last thing that I tried... In my original source code, only removing the root handle is also sufficient to make it work. The `FileNameLength` seems to be ignored (can also be set to 0). Only the `DELETE` right appears to be required for `f_handle`. – dyp Apr 09 '16 at 12:31
  • 2
    Yes, DELETE is only flag required when you create file handle, you could see it at: https://msdn.microsoft.com/en-us/library/windows/hardware/ff540344%28v=vs.85%29.aspx – Ing. Gerardo Sánchez Apr 09 '16 at 15:37
  • I cannt find any reason why RootDirectory must be NULL, but i guess: it works in kernel mode, or because FILE_RENAME_INFO is shared for SetFileInformationByHandle, NtSetInformationFile & ZwSetInformationFile, maybe it has some sense use RootDirectory != NULL in other functions – Ing. Gerardo Sánchez Apr 09 '16 at 15:40
  • 1
    Wow, it says why it needs `RootDirectory == NULL` in the description you've linked. I did not see that because I was navigating from `SetFileInformationByHandle`, where you end up at the [FILE_RENAME_INFO](https://msdn.microsoft.com/en-us/library/windows/desktop/aa364398%28v=vs.85%29.aspx) structure instead of the [FILE_RENAME_INFORMATION](https://msdn.microsoft.com/en-us/library/windows/hardware/ff540344%28v=vs.85%29.aspx) structure you've linked to. Many thanks! – dyp Apr 09 '16 at 15:57
  • Yes, but if you try to use **RootDirectory != NULL** it fails, even when manual says it must work – Ing. Gerardo Sánchez Apr 09 '16 at 16:18
  • Let me explain you very carefully my comments: if you walk inside SetFileInformationByHandle in a dissamsembly window, you find that functions calls NtSetInformationFile, if you search a little this last function uses FILE_RENAME_INFORMATION but its only a synonymous for FILE_RENAME_INFO, in fact SetFileInformationByHandle just pass this struct directly with any conversion to NtSetInformationFile. BECAUSE THAT you could use help related to FILE_RENAME_INFORMATION safely in order to understand FILE_RENAME_INFO. – Ing. Gerardo Sánchez Apr 11 '16 at 03:08
  • 2
    Non-null `RootDirectory` works just fine for relative/race free renames, but no `HANDLE` anywhere in the system can be open on that directory with the `DELETE` permission held. Yes, this sucks. But it is what it is. – Niall Douglas Sep 19 '17 at 01:06
  • On my computer (Win10 21H1) it only works with an *absolute* file name and RootDirectory == NULL. Using a relative file name doesn't work here with RootDirectory == NULL. – David Gausmann Feb 21 '22 at 17:48