1

Problem: I want to use FileOpenPicker to read a binary file over 4GB into std::vector in UWP

It'd also be great if I could use the default picker, but open it with fstream instead. I'm guessing this is easy to do and quite possible, but I'm having trouble figuring it out.

Background: The file I want to load is typically between 2GB and 4GB in size, and I haven't been able to figure out what sort of guarantee (if any) the typical UWP library options provide. I don't want to stream the data through a buffer, unless the buffer is large enough to fit the entire file.

Irrelevant aside: I feel like someone is going to ask about why I'd want to load a large file into memory instead of streaming it or buffering it, and it's because I need to perform a number of operations on all of the data over multiple iterations, and the disk I/O slows down the calculations a lot.

edit: example: I've tried a number of things. I'd love it if I had something like this:

Windows::Storage::Pickers::FileOpenPicker ^picker = ref new FileOpenPicker();
concurrency::create_task(picker->PickSingleFileAsync()).then([](StorageFile ^file) {
    /* some tests */
    std::experimental::filesystem::path data_file = to_string(file->Path);
    /* access file here */
}
/* or access file here, I don't care where as long as I can do it */

The issue with this implementation is that I cannot actually do anything with the file. It correctly returns the path to the file, but I cannot access it. I've also tried just returning the name and operating on that, but I still can't use fstream on the file. I suspect that's due to the restricted capabilities in UWP. https://learn.microsoft.com/en-us/windows/uwp/files/file-access-permissions I know it's not recommended, but short of just being able to load data into a vector via some msft pointers, I'd at least like to be able to get it via normal ISO C++ operations.

I've also tried implementing something similar to this (a sample from msft): https://github.com/Microsoft/Windows-universal-samples/blob/master/Samples/FileAccess/cpp/Scenario5_WriteAndReadAFileUsingAStream.xaml.cpp but those too fail to load the file. A few samples from that set don't build on visual studio, so I'm not sure if they are any good to check out anyway.

I was able to load a bmp more or less following the examples given, but I couldn't figure out how to load normal binary data using similar constructs. They have a lot of functionality for some image classes, but I don't have images.

I noticed this comment, but it just looked like an implementation issue: Files larger than 4GB need to be broken into multiple chunks to be loaded by LoadAsync.

Anyway, I figure that there has to be something as easy as this for UWP:

std::ifstream ifs(data_file, std::ios::binary | std::ios::ate);
/* find end of file, store in size, reserve memory for vector */
ifs.read(byte_vector.data(), size);

Edit 2: I've resorted to reading random blogs by MSFT people. https://blogs.windows.com/buildingapps/2016/11/28/standard-c-windows-runtime-cwinrt/

The suggestion in this blog was to do something like this:

FileOpenPicker picker;
picker.FileTypeFilter().Append(L".png");
picker.SuggestedStartLocation(PickerLocationId::PicturesLibrary);
auto file = co_await picker.PickSingleFileAsync();

Unfortunately, when I switched over and tried to do a WinRT app, the only thing available was FileOpenPickerUI, which is totally different from FileOpenPicker. The code above also coughs up on the "co_await" line.

Another random MSFT blog post was presented here: https://blogs.windows.com/buildingapps/2018/05/18/console-uwp-applications-and-file-system-access/ This individual reads a file into a string (I know, bad practice 101, but literally half of their documentation does the same thing.)

hstring text = FileIO::ReadTextAsync(file).get();

When I attempt to modify the above line, I can't get it to read into a vector.

I also decided to try doing something like this: https://stackoverflow.com/a/14383816/9824697, where I put the file retrieved from the picker (or attempt to?) into an IBuffer. This also didn't work for me.

EDIT 3: Ok, so I tried this (more or less from here: https://github.com/Microsoft/Windows-universal-samples/blob/6370138b150ca8a34ff86de376ab6408c5587f5d/Samples/Simple3DGameDX/cpp/Common/BasicReaderWriter.cpp)

create_task(picker->PickSingleFileAsync()).then([=](StorageFile ^picked) {
/* some error checking */
CREATEFILE2_EXTENDED_PARAMETERS extendedParams = { 0 };
extendedParams.dwSize = sizeof(CREATEFILE2_EXTENDED_PARAMETERS);
extendedParams.dwFileAttributes = FILE_ATTRIBUTE_NORMAL;
extendedParams.dwFileFlags = FILE_FLAG_SEQUENTIAL_SCAN;
extendedParams.dwSecurityQosFlags = SECURITY_ANONYMOUS;
extendedParams.lpSecurityAttributes = nullptr;
extendedParams.hTemplateFile = nullptr;

String ^Lame = picked->Path;

Wrappers::FileHandle file(CreateFile2(Lame->Data(),
    GENERIC_READ,
    FILE_SHARE_READ,
    OPEN_EXISTING,
    &extendedParams));

FILE_STANDARD_INFO file_info = { 0 };
GetFileInformationByHandleEx(file.Get(), FileStandardInfo, &file_info, sizeof(file_info));
Platform::Array<byte> ^file_data = ref new Platform::Array<byte>(file_info.EndOfFile.LowPart);

ReadFile(file.Get(), file_data->Data, file_data->Length, nullptr, nullptr);

But in the end, "file_data->Length" is always 0, and there's no data in file_data.

  • 1
    What have you tried? Can you show some code? – Jesper Juhl May 21 '18 at 19:19
  • @JesperJuhl, done! All I want to do is use the file picker to get a binary file and read it into a std::vector. I apparently need to use FileOpenPicker since UWP restricts what files and directories I have access to. –  May 22 '18 at 13:57
  • You can use `CreateFile2` followed by `ReadFile`. Best not to do this from your UI thread, or the system will probably terminate you for being unresponsive to input. Other notes: The UWP samples repo requires the April 2018 update to Visual Studio. You need the `/await` compiler switch to use `co_await`. Your third code block is written in C++/winrt, not C++/CX. – Raymond Chen May 22 '18 at 14:10

0 Answers0