1

When the Xbox 360 console formats a 1gb USB device, it adds 978mb of data to it in just 20 seconds. I can see the files on the USB and they are that size.

When I copy a file of the same length in Windows, it takes 6 minutes.

Maybe it is because Windows reads/writes, but the 360 just writes?

Is there a way to create large files like that on a USB with that kind of performance? The files can be blank, of course. I need this writing performance for my application.

Most of the cmd tools I have tried have not had any noticeable performance gains.

Eaton
  • 1,310
  • 2
  • 15
  • 31

4 Answers4

3

It would appear that the 360 is allocating space for the file and writing some data to the file, but is otherwise leaving the rest of the file filled with whatever data was there originally (so-called "garbage data"). When you copy a file of the same size to the drive, it is writing all 978MB of, which is a different scenario and is why it takes so much longer.

Spencer Hakim
  • 1,543
  • 9
  • 19
  • Hm, that would make sense because when looking at the Xbox-created file in a hex editor, I can see parts of old files on my USB. Pictures, music, etc. – Eaton Nov 29 '10 at 15:05
  • Yup, that's the garbage data. Also, the 360 may be faster than fsutil/contig tools Philip Rieck mentioned due to some driver and OS optimizations, since they know that all the systems will have the same (or one of only a few) USB chipsets. – Spencer Hakim Nov 29 '10 at 15:10
  • I see. Do you know of a good way I would be able to create a system that will at least be faster than Windows? – Eaton Nov 29 '10 at 15:16
  • 1
    If you create a 0 byte file and then use this function through Interop, I think it'll do what you want. http://msdn.microsoft.com/en-us/library/aa365544(VS.85).aspx – Spencer Hakim Nov 29 '10 at 15:26
  • Wait, no, that's not right. That function can only decrease the the file size. Use CreateFile, then SetFilePointer to set the new size, and then finally SetEndOfFile. – Spencer Hakim Nov 29 '10 at 15:38
  • Windows completely locks up when I do it. It sets the file size, I stop debugging, and it takes about 6 minutes to stop debugging. And when I try to refresh the view on Windows Explorer, it locks up too. – Eaton Nov 29 '10 at 17:13
  • Code: (Tried to force exit upon completion, but no avail) http://pastie.org/1332822 – Eaton Nov 29 '10 at 17:14
  • You need to close the file handle, although that shouldn't be causing a lockup (unless the drivers are being weird?) – Spencer Hakim Nov 29 '10 at 17:20
  • Think the lockup was a glitch. I added CloseHandle, but it seems to be taking forever to close. When I view the USB in explorer, I see the file and it looks like the job is done. However, my program is still closing the handle and the light on the USB is blinking. – Eaton Nov 29 '10 at 17:24
  • Hmmm, if it's blinking then that means something is reading/writing to the drive. I'll post if I figure this out. – Spencer Hakim Nov 29 '10 at 17:35
  • Yeah, it seems that is it writing the entire file since it is taking the same amount of time as a normal 1gb file copy. Thanks a lot for the help, by the way. – Eaton Nov 29 '10 at 17:37
  • @Eaton Ok, so, after digging around, you need to call `SetFileValidData` after `SetEndOfFile`. Make sure to pay attention to the `SE_MANAGE_VOLUME_NAME` requirement. – Spencer Hakim Nov 29 '10 at 17:44
  • Updated code, still not working as expected: http://pastie.org/1332949 And what do I need to do to meet that requirement? – Eaton Nov 29 '10 at 18:00
  • @Eaton You need to be running Win2k or XP at a minimum (if you somehow don't meet this requirement, I will be utterly baffled), and `SE_MANAGE_VOLUME_NAME` refers to a specific security privilege. You may need to run your app as an admin to receive that privilege, I'm not sure. – Spencer Hakim Nov 29 '10 at 18:06
  • I am running it as admin, but let me share another finding. When looking at the USB in WinHex and going to the allocation table, it is filled. Probably with the entries for the Xbox file put on. I am also guessing that it left the file area untouched. So basically, I think I just need to add the data file to the directory listing and fill the allocation table accordingly. But I guess I would need to code a FAT32 class from scratch to do that. )-: Unless you can think of a better way. – Eaton Nov 29 '10 at 18:22
  • Wow, it didn't even occur to me that the USB drive was FAT32. The stuff I told you only works on NTFS. So, yeah, your only real option is directly playing around with the allocation table. – Spencer Hakim Nov 29 '10 at 18:26
  • Ah, ok. Well, thank you so much for the help and the idea about the garbage data. That made the strange data I saw make sense. I'll mark your answer as the answer. – Eaton Nov 29 '10 at 18:34
  • @Eaton No problem, glad I could help answer your questions. – Spencer Hakim Nov 29 '10 at 18:40
2

Most likely the 360 is not sending 978mb of data to the usb stick, but is instead creating an empty file of size 978mb - yours takes longer because rather than simply sending a few KB to alter the file system information, you are actually sending 978mb of data to the device.

You can do something similar (create an empty file of fixed size) on windows with fsutil or Sysinternals "contig" tool: See Quickly create large file on a windows system? - try this, and you'll see that it can take much less than 20 seconds (I would guess that the 360 is sending some data, as well as reserving space for more). Note that one of the answers shows how to use the windows API to do the same thing, as well as a python script.

Community
  • 1
  • 1
Philip Rieck
  • 32,368
  • 11
  • 87
  • 99
  • I tried fsutil, still took about 5 minutes to create the 978mb file. – Eaton Nov 29 '10 at 15:01
  • Try the Sysinternals "Contig" tool: http://technet.microsoft.com/en-us/sysinternals/bb897428.aspx – Philip Rieck Nov 29 '10 at 15:01
  • Also, when looking at the Xbox-created file in a hex editor, I can see parts of old files on my USB. Pictures, music, etc. – Eaton Nov 29 '10 at 15:03
  • @Eaton On my machine, `Contig` takes no time to create a 50GB file on the local file system, and ~7 seconds on a USB stick (an older one at that) – Philip Rieck Nov 29 '10 at 15:08
  • I used this command: "contig -n file 68709120" and it took approximately 30 seconds to display the completed message. – Eaton Nov 29 '10 at 15:15
  • @Eaton - Finding a "free giveaway" usb stick, creating a file with `contig -n test.fil 687091200` also took around 1 min. It seems that fsutil and contig must do something that some usb sticks do very slowly (seek to end? mark blocks used?). It looks like the 360 must directly mess with the file system info, or use a method tuned to the characteristics of usb sticks. – Philip Rieck Nov 29 '10 at 15:29
0

Could it be that the 360 is just doing some direct filesystem header manipulation? If a blank file is fine for you maybe you could try that?

dain
  • 6,475
  • 1
  • 38
  • 47
  • I want to be able to create blank files. The thing is, it slow using the cmd apps and C# programs I have made for it. I want to know if there is a faster way. – Eaton Nov 29 '10 at 14:57
0

It is all dependent on the throughput of the usb drive. You will need a high end usb such as the following: this list

Woot4Moo
  • 23,987
  • 16
  • 94
  • 151