1

Hello fellow programmers.

I'm trying to dump the contents of the USN Journal of a NTFS partition using WinIoCtl functions. I have the *USN_JOURNAL_DATA* structure that tells me that it has a maximum size of 512 MB. I have compared that to what fsutil has to say about it and it's the same value.

Now I have to read each entry into a *USN_RECORD* structure. I do this in a for loop that starts at 0 and goes to the journal's maximum size in increments of 4096 (the cluster size). I read each 4096 bytes in a buffer of the same size and read all the USN_RECORD structures from it.

Everything is going great, file names are correct, timestamps as well, reasons, everything, except I seem to be missing some recent records. I create a new file on the partition, I write something in it and then I delete the file. I run the app again and the record doesn't appear. I find that the record appears only if I keep reading beyond the journal's maximum size. How can that be?

At the moment I'm reading from the start of the Journal's data to the maximum size + the allocation delta (both are values stored in the *USN_JOURNAL_DATA* structure) which I don't believe it's correct and I'm having trouble finding thorough information related to this.

Can someone please explain this? Is there a buffer around the USN Journal that's similar to how the MFT works (meaning it's size halves when disk space is needed for other files)?

What am I doing wrong?

Sebastian-Laurenţiu Plesciuc
  • 2,413
  • 1
  • 24
  • 41

1 Answers1

3

That's the expected behaviour, as documented:

MaximumSize

The target maximum size for the change journal, in bytes. The change journal can grow larger than this value, but it is then truncated at the next NTFS file system checkpoint to less than this value.

Instead of trying to predetermine the size, loop until you reach the end of the data.

If you are using the FSCTL_ENUM_USN_DATA control code, you have reached the end of the data when the error code from DeviceIoControl is ERROR_HANDLE_EOF.

If you are using the FSCTL_READ_USN_JOURNAL control code, you have reached the end of the data when the next USN returned by the driver (the DWORDLONG at the beginning of the output buffer) is the USN you requested (the value of StartUsn in the input buffer). You will need to set the input parameter BytesToWaitFor to zero, otherwise the driver will wait for the specified amount of new data to be added to the journal.

Community
  • 1
  • 1
Harry Johnston
  • 35,639
  • 6
  • 68
  • 158
  • Thank you. Haven't thought of that :) – Sebastian-Laurenţiu Plesciuc Feb 26 '13 at 21:50
  • Actually, DeviceIoControl doesn't return a LSTATUS, it returns a BOOL indicating if it failed or not. However I looped until GetLastError() returned ERROR_HANDLE_EOF but it didn't return that code, it just kept looping and showing recent changes to the USN journal. Kinda like it would read the last X bytes over and over again. So to stop the loop eventually you have to loop until the returned buffer doesn't change, but this can't guarantee that we read the entire USN Journal. Looping until the offset gets around the MaximumSize + AllocationDelta seems to work though. – Sebastian-Laurenţiu Plesciuc Feb 28 '13 at 06:18
  • 1
    What IO control code are you using? My code, using `FSCTL_ENUM_USN_DATA`, definitely receives ERROR_HANDLE_EOF when it reaches the end of the MFT. See http://stackoverflow.com/a/7459109/886887 – Harry Johnston Feb 28 '13 at 23:15
  • 1
    (Note that when dealing with the Windows API, "F returns error code X" is often shorthand for "F returns FALSE and GetLastError() returns error code X"; that's just one of those things that gets taken for granted.) – Harry Johnston Feb 28 '13 at 23:22
  • I used FSCTL_READ_USN_JOURNAL. Thank you for the update. I constantly update the requested USN so that I get all USN entries, but I update it with the USN record number that is in the buffer, so the last entry would be requested again and again. I also failed to make the distinction between the FSCTL_READ_USN_JOURNAL and FSCTL_ENUM_USN_DATA. Thank you for clearing that up for me. :) – Sebastian-Laurenţiu Plesciuc Mar 04 '13 at 06:44