0

First, here are a couple links i looked at...

Read and write hard disk sector directly and efficiently

Read specific sector on hard drive using C language on windows

I'm trying to do almost the same thing. What I'm having trouble with is reading the device multiple times so I can store the read bytes from a DEVICE (USB) to writing them into a FILE.

This is what I am trying to do...

  1. declare variables
  2. initialize variables
  3. SetFilePointer()
  4. ReadFile()
  5. output the read bytes to a file
  6. use ReadFile() to get more bytes
  7. output the read bytes to same file again
  8. repeat 6 and 7 (actually just repeating 4 and 5)

This doesn't seem to be working. I would like to read x amount of bytes and store those values into a file then read more and store those values in the same file as last time. I would like it to repeat the process until it reads to the end of the device. I would like this program to work on any size of device. If I can put it in a loop then I can read and write infinite sized devices.

This is a how to read/write so I would like to do this in reverse as well. Read the file for values then write them to the device.

I'm using a 128MB USB. It contains 131858432 bytes. If any more info is needed then I will post it if I have it.

My code:

#include <windows.h>
#include <stdio.h>

int main(int argc, char ** argv)
{
    BYTE sector[0x400] = {0};
    DWORD bytesRead;
    HANDLE device = NULL;
    int numSector = 1;
    int maxRead = 1;
    FILE *readChar = fopen("G:\\usb_128MB_Dec2.txt", "w+");
    
    device = CreateFile("\\\\.\\L:",                        // Drive to open
                        GENERIC_READ|GENERIC_WRITE,         // Access mode
                        FILE_SHARE_READ|FILE_SHARE_WRITE,   // Share Mode
                        NULL,                               // Security Descriptor
                        OPEN_EXISTING,                      // How to create
                        0,                                  // File attributes
                        NULL);                              // Handle to template
    
    if(device == INVALID_HANDLE_VALUE)
    {
        printf("CreateFile: %u\n", GetLastError());
        system("pause");
        return 1;
    }
    
    //  set the file pointer for first time
    SetFilePointer(device, numSector, NULL, FILE_BEGIN);

    // Edit 1. Comment 2.
    if(GetLastError() != NO_ERROR)
    {
        printf("GetLastError: %d\n", GetLastError());
        goto end;       //  end: is before closing files or handles at end of main()
    }
    
    //  read device for maxRead number of bytes and store the reading into a file
    ReadFile(device, sector, maxRead, &bytesRead, NULL);
    fprintf(readChar, "%d\n", sector[0]);
    
    //  This part of code does not act like expected
    SetFilePointer(device, numSector, NULL, FILE_CURRENT);
    if(!ReadFile(device, sector, maxRead, &bytesRead, NULL))
         printf("err\n");
    else
        fprintf(readChar, "%d", sector[0]);
    
end:    //  Edit 1. Comment 2.
    CloseHandle(device);
    fclose(readChar);
    system("pause");
    return 0;
}

Screen ouput:

GetLastError: 87

File output:

Nothing is in the file.

The file should contain an 8 bit decimal value instead of nothing or a 0.

Edit 1:

87 means INVALID PARAMETER.

Your reads have to be sector aligned so you can't seek to offset 1, but only 0, sector_size, 2sector_size, ..., nsector_size, Your reads have also be in multiplies of sector size, and your memory buffer has to be aligned to sector_size.

Sector size could be retrieved with GetDiskFreeSpace and aligned memory can be obtained with VirtualAlloc.

Maybe you should check also FSCTL_LOCK_VOLUME and FSCTL_DISMOUNT_VOLUME.

Here's the other code that works for one time reading of the device

#include <windows.h>
#include <stdio.h>

int main(int argc, char ** argv)
{
    BYTE sector[0x200] = {0};
    DWORD bytesRead;
    HANDLE device = NULL;
    int numSector = 1;
    int maxRead = 0x200;
    long long int i;
    FILE *readChar = fopen("G:\\usb_128MB_Dec.txt", "w+");

    device = CreateFile("\\\\.\\L:",                        // Drive to open
                        GENERIC_READ|GENERIC_WRITE,         // Access mode
                        FILE_SHARE_READ|FILE_SHARE_WRITE,   // Share Mode
                        NULL,                               // Security Descriptor
                        OPEN_EXISTING,                      // How to create
                        0,                                  // File attributes
                        NULL);                              // Handle to template

    if(device == INVALID_HANDLE_VALUE)
    {
        printf("CreateFile: %u\n", GetLastError());
        system("pause");
        return 1;
    }

    SetFilePointer(device, numSector, NULL, FILE_BEGIN);
    
    if (!ReadFile(device, sector, maxRead, &bytesRead, NULL))
    {
        printf("ReadFile: %u\n", GetLastError());
        goto end;
    }
    else
    {
        printf("Success!\n");
    }
    
    if(readChar == NULL)
    {
        printf("Did not open file. Exit 2.");
        goto end;
    }
    
    for(i = 0; i < maxRead - 1; i++)
    {
        fprintf(readChar, "%d\n", sector[i]);
    }
    fprintf(readChar, "%d", sector[i]);     // so the previous loop wont add \n to the last read wanted
    
end:
    CloseHandle(device);
    fclose(readChar);
    system("pause");
    return 0;
}

File contents:

235
88
...

Each byte read is stored in decimal value on a new line.

So it may be better understood what I'm trying to do, here it is in code:

//  What I want to do..
//  This part works
SetFilePointer(device, numSector, NULL, FILE_BEGIN);
ReadFile(device, sector, maxRead, &bytesRead, NULL);

for(i = 0; i < size_of_device - 0x200; i += 512)
{
    for(j = 0; j < maxRead; j++)
    {
        fprintf(readChar, "%d\n", sector[j]);
    }
    
    // stops working
    SetFilePointer(device, numSector, NULL, FILE_CURRENT);
    ReadFile(device, sector, maxRead, &bytesRead, NULL);
}

for(j = 0; j < maxRead - 1; j++)
{
    fprintf(readChar, "%d\n", sector[j]);
}
fprintf(readChar, "%d", sector[j]);
//  .. end of what i want to do

Edit 2: Working on reading multiple times now.

#include <windows.h>
#include <stdio.h>

int main(int argc, char ** argv)
{
    BYTE sector[0x200] = {0};
    DWORD bytesRead;
    HANDLE device = NULL;
    //int numSector = 512;      //  original value was 1 not 512 but variable is not needed
    int maxRead = 512;
    int i, j, k = 0, l;             //  loop variables
    
    FILE *readChar = fopen("G:\\wii u hdd image\\usb_128MB_Dec3.txt", "w+");
    
    if(readChar == NULL)
    {
        printf("Error creating file.\n");
        goto end;
    }
    
    device = CreateFile("\\\\.\\L:",                        // Drive to open
                        GENERIC_READ|GENERIC_WRITE,         // Access mode
                        FILE_SHARE_READ|FILE_SHARE_WRITE,   // Share Mode
                        NULL,                               // Security Descriptor
                        OPEN_EXISTING,                      // How to create
                        0,                                  // File attributes
                        NULL);                              // Handle to template
    
    //  If device does not contain a handle value
    if(device == INVALID_HANDLE_VALUE)
    {
        printf("Error. GetLastError: %u\n", GetLastError());
        goto end;
    }
    
    for(i = 0; i < maxRead*503; i++)    //  maxRead * 503 = 257536
    {
        //  If ReadFile() fails it will exit the program without adding a '\n' to the readChar file.
        if(!ReadFile(device, sector, maxRead, &bytesRead, NULL))
        {
            printf("Error of ReadFile(). GetLastError(): %u\n", GetLastError());
            goto end;
        }
        
        //  If this is the first time through the loop then '\n' won't be added to the readChar file.
        if(i != 0)
        {
            fprintf(readChar, "\n");
            system("cls");
            printf("%.2f%%\n", (i / 257536));
        }
        
        //  Runs for 511 times. Then prints the 512th decimal value after the loop.
        for(j = 0; j < maxRead - 1; j++)
        {
            fprintf(readChar, "%d\n", sector[j]);
        }
        fprintf(readChar, "%d", sector[j]);
    }
        
end:
    CloseHandle(device);
    fclose(readChar);
    system("pause");
    return 0;
}

Edit 3:

This question has not been answered in another post.

Daniel Vaughn
  • 13
  • 2
  • 7
  • Reading MSDN for `ReadFile`: "If the function fails, or is completing asynchronously, the return value is zero (FALSE). To get extended error information, call the GetLastError function." - I suggest you try that – YePhIcK Oct 05 '17 at 18:46
  • When you call `SetFilePointer(device, 1, NULL, FILE_BEGIN)` what are you trying to do? The naming convention makes it look like you want to set the file pointer to the start of the second sector (sector 1) but what the call is actually trying to do is to set the file pointer to the second byte of sector zero. Similarly, when you call `ReadFile` with `maxRead` set to 1 it isn't obvious to me whether you wanted to read a single byte or an entire sector. – Harry Johnston Oct 05 '17 at 22:24
  • Note also that after calling `SetFilePointer` you should only call `GetLastError` if the return value was `INVALID_SET_FILE_POINTER`. And your code doesn't check for errors consistently. The call to `SetFilePointer` in the "one time reading" version of the code is probably failing, you just didn't notice. – Harry Johnston Oct 05 '17 at 22:30
  • SetFilePointer(device, numSector, NULL, FILE_BEGIN); I'm trying to get it to set the pointer to the beginning of the file or device. I have removed it and just used ReadFile() multiple times to get the desired reading. ReadFile() will move the pointer a number of bytes read. So after the first read of 512 bytes and storing it in FILE *readChar, ReadFile() will continue from where it was when used again. @HarryJohnston – Daniel Vaughn Oct 06 '17 at 20:45
  • If you want to move to the beginning, that would be `SetFilePointer(device, 0, NULL, FILE_BEGIN);` – Harry Johnston Oct 07 '17 at 00:26
  • Does your new code work? Do you still have a problem? – Harry Johnston Oct 07 '17 at 00:26
  • I made a small change after posting Edit 2. It was `printf("%.2f%%", (i / 257536));` I changed it to `printf("%.2f%%", ((float) i / (float)257536 * 100.0));`. This way I could see the percentage of completion while it was working. It seems to have worked now. Guess I didn't need SetFilePointer(). I've tested `SetFilePointer(device, 1, NULL, FILE_BEGIN);`. It does the same as `SetFilePointer(device, 0, NULL, FILE_BEGIN);`. It only changes where it sets the file pointer if I use 512 or 1024 or any other number that's a multiple of 512. @HarryJohnston – Daniel Vaughn Oct 07 '17 at 09:23
  • Yes, that's the expected behaviour, as explained in Daniel's answer. – Harry Johnston Oct 07 '17 at 22:20
  • With the way daniel worded it, this was not clear. Also its not even an answer to my problem. My question has not been answered on another post either. An answer to this question would involve more than just a line of code. @HarryJohnston – Daniel Vaughn Oct 08 '17 at 19:31
  • I'll be happy to reopen the question, but at present it isn't at all clear what you're asking. Your original code had two obvious problems: you were using SetFilePointer incorrectly and you were only reading one byte at a time. – Harry Johnston Oct 08 '17 at 20:08
  • So far as I can see you have corrected both those problems (as per Daniel's answer) and you said in the comments "it seems to have worked now". If you still have a problem, you haven't explained what it is. In any case, your *original* problem (error code 87 returned from SetFilePointer) is definitely answered by the duplicates. – Harry Johnston Oct 08 '17 at 20:14
  • ... the problem is how to read/write hdd/usb bytes in c. @HarryJohnston – Daniel Vaughn Oct 08 '17 at 20:32
  • Heres what i want in steps. 1.) Read n number of bytes from anywhere on device. 2.) Store read byte values in a file. Stored byte values must be in decimal. 3.) Read file of stored byte values. 4.) Write byte values from the file to anywhere on the device. @HarryJohnston – Daniel Vaughn Oct 08 '17 at 20:42
  • 1
    If you're expecting someone to write your code for you, you're on the wrong site. – Harry Johnston Oct 08 '17 at 20:44
  • I dont expect someone to write the code for me unless its a line or two, for example... this is how to print to the screen `printf("hello");`. Meaning the person is saying "how to" and shows an example of code. That has been done in previous comments. I guess i would be on the wrong site though seeing that no one can understand and give an appropriate answer to the problem. There has been 0 answers to the actual problem. Just error correcting of minor problems to the big picture. If this is going to cause removal of the thread then i guess that's ok. No one is really helping anyway. – Daniel Vaughn Oct 08 '17 at 21:02
  • Well, I'm sorry, but we can't tell you how to fix whatever problem you're having making the code do what you want it to unless you're willing to explain *what that problem is*. Your original question said that the problem was that you were getting error 87, and we've explained how to stop that from happening. If there's some other problem, I have no idea what it is. The code as posted ("edit 2") looks as if it meets your specifications. – Harry Johnston Oct 08 '17 at 21:11
  • See [illusion of transparency](http://lesswrong.com/lw/ke/illusion_of_transparency_why_no_one_understands/): "We always know what we mean by our words, and so we expect others to know it too. Reading our own writing, the intended interpretation falls easily into place, guided by our knowledge of what we really meant. It's hard to empathize with someone who must interpret blindly, guided only by the words." – Harry Johnston Oct 08 '17 at 21:14
  • You're right. Sorry. I will try to explain it better soon (my next day off work). @HarryJohnston – Daniel Vaughn Oct 09 '17 at 06:50

1 Answers1

3

87 means INVALID PARAMETER.

Your reads have to be sector aligned so you can't seek to offset 1, but only 0, sector_size, 2*sector_size, ..., n*sector_size, Your reads have also be in multiplies of sector size, and your memory buffer has to be aligned to sector_size.

Sector size could be retrieved with GetDiskFreeSpace and aligned memory can be obtained with VirtualAlloc.

Maybe you should check also FSCTL_LOCK_VOLUME and FSCTL_DISMOUNT_VOLUME.

Edit

Because direct reading (or writing) isn't buffered, if you want to operate on smaller sizes then sector size, you have to handle buffering yourself.

This code implements reading single bytes from arbitrary places using one sector buffering scheme. Disclaimer: Needs through testing. Beware of possible bugs.

#include <windows.h>

typedef __int64 int64;

struct MyDevice
{
    HANDLE  handle;
    DWORD   sector_size;
    int64   current_sector_position;
    void*   sector_buffer;
};

BOOL OpenMyDevice( struct MyDevice* device, const char* name )
{
    device->current_sector_position = -1;
    device->handle = INVALID_HANDLE_VALUE;
    device->sector_buffer = 0;

    {
        DWORD   bytes_per_sector, unused1, unused2, unused3;
        // GetDiskFreeSpace doesn't like "\\.\".
        const char* name2 = name;
        if ( strncmp( name, "\\\\.\\", 4 ) == 0 )
            name2 = name + 4;
        // For comaptibility reasons we will get logical sector size here.
        // For Vista+ it would be better to use DeviceIoControl with IOCTL_STORAGE_QUERY_PROPERTY.
        if ( !GetDiskFreeSpace( name2, &unused1, &bytes_per_sector, &unused2, &unused3 ) )
            return FALSE;
        device->sector_size = bytes_per_sector;
    }

    device->sector_buffer = VirtualAlloc( 0, device->sector_size, MEM_COMMIT, PAGE_READWRITE );
    if ( !device->sector_buffer )
        return FALSE;

    device->handle = CreateFile(
            name,
            GENERIC_READ,
            FILE_SHARE_READ | FILE_SHARE_WRITE,
            0,
            OPEN_EXISTING,
            FILE_FLAG_NO_BUFFERING, // Can be zero, but in most cases it assumed by Windows.
            0 );
    if ( device->handle == INVALID_HANDLE_VALUE )
    {
        VirtualFree( device->sector_buffer, 0, MEM_RELEASE );
        device->sector_buffer = 0;
        return FALSE;
    }

    return TRUE;
}

// Call only when OpenDevice was successful.
void CloseMyDevice( struct MyDevice* device )
{
    CloseHandle( device->handle );
    device->handle = INVALID_HANDLE_VALUE;
    VirtualFree( device->sector_buffer, 0, MEM_RELEASE );
    device->sector_buffer = 0;
}

BOOL LoadMyDeviceSector( struct MyDevice* device, int64 byte_offset )
{
    // Calculate sector position.
    int64 sector_position = ( byte_offset / device->sector_size ) * device->sector_size;

    if ( sector_position == device->current_sector_position )
        // No need to load, it is in buffer
        return TRUE;

    {
        LARGE_INTEGER   li;
        li.QuadPart = sector_position;
        if ( SetFilePointerEx( device->handle, li, 0, FILE_BEGIN ) )
        {
            DWORD   read;
            if ( ReadFile( device->handle, device->sector_buffer, device->sector_size, &read, 0 ) )
            {
                if ( read == device->sector_size )
                {
                    device->current_sector_position = sector_position;
                    return TRUE;
                }
                // else Hmmm. Maybe this is read beyond EOF?
            }
        }
    }

    // Cant guarantee that device->sector_buffer contains valid data.
    device->current_sector_position = -1;
    return FALSE;
}

BOOL LoadNextMyDeviceSector( struct MyDevice* device )
{
    DWORD   read;
    device->current_sector_position = -1;
    if ( ReadFile( device->handle, device->sector_buffer, device->sector_size, &read, 0 ) )
    {
        if ( read == device->sector_size )
            return TRUE;
        // else Hmmm. Maybe this is read beyond EOF?
    }
    return FALSE;
}

BOOL SetMyDevicePos( struct MyDevice* device, int64 sector_aligned_byte_offset )
{
    LARGE_INTEGER   li;
    li.QuadPart = sector_aligned_byte_offset;
    device->current_sector_position = -1;
    return SetFilePointerEx( device->handle, li, 0, FILE_BEGIN );
}

int GetMyDeviceByte( struct MyDevice* device, int64 offset )
{
    if ( LoadMyDeviceSector( device, offset ) )
    {
        // Calculate position in sector buffer.
        int64   offset_in_sector = offset - ( offset / device->sector_size ) * device->sector_size;
        return ((unsigned char*)( device->sector_buffer ))[ offset_in_sector ];
    }
    return -1;
}

BOOL GetMyDeviceBytes( struct MyDevice* device, int64 byte_offset, void* dst_buffer, int64 count )
{
    char* dst = (char*) dst_buffer;
    int64 sector_position = ( byte_offset / device->sector_size ) * device->sector_size;
    int64 start = byte_offset - sector_position;    // First loop pass can be unaligned!
    while ( count > 0 )
    {
        if ( LoadMyDeviceSector( device, byte_offset ) )
        {
            int64 chunk = device->sector_size - start;
            if ( chunk > count )
                chunk = count;
            // chunk <= sector_size so conversion to int isn't harmful.
            memcpy( dst, ((char*)(device->sector_buffer)) + start, (int)chunk );
            dst += chunk;
            byte_offset += chunk;
            count -= chunk;
            start = 0;  // From now loop would be always sector_size aligned.
        }
        else
            return FALSE;
    }
    return TRUE;
}

int main( int argc, char* argv[] )
{
    struct MyDevice device = { INVALID_HANDLE_VALUE };
    if ( OpenMyDevice( &device, "\\\\.\\K:" ) )
    {
        // #1: Get one byte from device.
        {
            int byte = GetMyDeviceByte( &device, 11111 );
            if ( byte >= 0 )
            {
            }
        }
        // #2: Get multiple bytes from device.
        {
            char buff[1000];
            if ( GetMyDeviceBytes( &device, 111111, buff, 1000 ) )
            {
            }
        }
        // #3: Scan 100 sectors beginning from sector 111 for byte 155.
        {
            if ( SetMyDevicePos( &device, 111*device.sector_size ) )
            {
                int i, j;
                for ( i = 0 ; i < 100 ; ++i )
                {
                    if ( !LoadNextMyDeviceSector( &device ) )
                        break;
                    for ( j = 0 ; j < (int)device.sector_size ; ++j )
                    {
                        int byte = ((unsigned char*)( device.sector_buffer ))[ j ];
                        if ( byte == 155 )
                        {
                            // FOUND!
                        }
                    }
                }
            }
        }
        CloseMyDevice( &device );
    }
}

GetMyDeviceByte and GetMyDeviceBytes are rather random seek oriented for small byte tranfers. If someone needs to trasfer large amounts of data in sequential order it is a lot faster to SetMyDevicePos and LoadNextMyDeviceSector like #3 in main.

I would prefer to write this code in C++.

Daniel Sęk
  • 2,504
  • 1
  • 8
  • 17
  • What include file is used for `FSCTL_LOCK_VOLUME` and `FSCTL_DISMOUNT_VOLUME`. My compiler says it's undeclared when i try using it. – Daniel Vaughn Oct 08 '17 at 19:13
  • `WinIoCtl.h` but it should be included by `windows.h`. – Daniel Sęk Oct 08 '17 at 19:29
  • When i try locking and unlocking the drive, it seems to unpartition the drive. I have to use device manager to reallocate it then reformat the drive. In the end it has less space than before. I put it in my wii u to format it then have to put it back in the pc to format once again to get all available space. – Daniel Vaughn Oct 09 '17 at 00:48
  • You are doing something wrong. Maybe writing to sectors with low numbers and destroying filesystem. I have edited my answer and added sample code showing how you can implement simple buffering scheme.. – Daniel Sęk Oct 09 '17 at 08:32