Windows File Systems?

Question

I have finished developing an archive format in managed C# that is way too flexible to be just an archive. I wish to use it as a file system.

It is well organized, very fast, and has both low-level and high-level API functions. It supports on-the-fly encryption and compression, password protection, Windows Explorer-specific file metadata (such as creation/modification/access time and attributes), 2^63 - 1 size HDDs, etc.

If it is possible to make Windows recognize and use my filesystem, I would learn Visual C++ just for the sake of implementing it. Are custom file systems even supported in Windows? (7 is a must, others are optional.) If they are, how do I make/implement them?

If I have to use some 3rd party library, it must be free. I don't want to use CallbackFileSystem because it's not free.

Maybe some readers are curious why I chose not to make my own driver. The reason is signing. Drivers, to work well on 64bit systems (at least my Windows 7) must be signed.
Now, I don't have the money to buy digital certificates from trusted sources... So no drivers for me... I use x64 Windows 7 on most of my machines so it would be an enormous waste to write the driver for x86...

I would assume that the filesystem(engine) is core technology and not easily replaceable. — Julius F, Jul 22 '11 at 11:37
Yes, http://msdn.microsoft.com/en-us/windows/hardware/gg462968 also look to the guys are OSR, who have toolkits and training. — kenny, Jul 22 '11 at 11:38
Sure. You're asking to make what's essentially a [Filesystem in Userspace](http://en.wikipedia.org/wiki/Filesystem_in_Userspace). The Windows equivalent would be an [Installtable File System](http://msdn.microsoft.com/en-us/windows/hardware/gg463062). — In silico, Jul 22 '11 at 11:40
@daemonfire There's a Microsoft update on Windows XP that adds exFAT to it's file systems... Both for formatting and I/O. — Vercas, Jul 22 '11 at 11:41
Whats wrong with a callback FS? It's not just that you don't know Visual C++, you also have to learn kernel-mode driver development, which is a much steeper and significant learning curve. — Joe, Jul 22 '11 at 11:41
@Joe CallbackFileSystem is a commercial product. 1. I am underaged - cannot purchase anything legally; 2. I am poor as heck; 3. It's darn expensive... — Vercas, Jul 22 '11 at 11:43
@Vercas: Be warned that C++ isn't trivial, and working with operating systems at such a low level is even less trivial. I recommend that you learn proper C++ first. [Here's a list of introductory C++ books to get started](http://stackoverflow.com/questions/388242/the-definitive-c-book-guide-and-list). — In silico, Jul 22 '11 at 11:44
@In silico Thank you but I know C++. I actually meant interfacing between "managed" and "unmanaged" code. — Vercas, Jul 22 '11 at 11:47
@Vercas: I'm sorry, when you said "I would learn Visual C++" I thought you meant that had no prior experience with C++. — In silico, Jul 22 '11 at 11:48
Thanks, Vercas. You can look at http://dokan-dev.net/en/ though last time I played with it it was not entirely stable. — Joe, Jul 22 '11 at 14:28
possible duplicate of [usermode file system](http://stackoverflow.com/questions/651171/usermode-file-system) — Ben Voigt, Jul 22 '11 at 14:35
@Joe I have been using Dokan for much time and I overcame all the bugs, even the "Eject" button in Removable Drives which plainly doesn't work - I have replaced it with my own Shell Extension. — Vercas, Jul 22 '11 at 16:40
@Ben Voigt It's **not** a duplicate. This is totally different and **not** in user mode. — Vercas, Jul 22 '11 at 16:56
@Vercas: C# is in user mode. Or are you developing for Singularity? — Ben Voigt, Jul 22 '11 at 17:15

score 2 · Accepted Answer · answered Jul 22 '11 at 11:41

2

Well, it is possible, once I have tried this approach. I've based on these samples: http://www.acc.umu.se/~bosse/

p.s. Also you don't need file system, you need a driver

answered Jul 22 '11 at 11:41

Dewfy

23,277
13
73
121

Best answer on any of my questions! I just love learning from code! It is going to take a while until I look at all the samples. xD – Vercas Jul 22 '11 at 11:50
Looking at http://www.acc.umu.se/~bosse/ntifs.html I understand that developing the driver will take longer than developing the FS... – Vercas Jul 22 '11 at 12:07
@Vercas - "A journey of thousand miles begins with a single step." It is not difficult as seen, a lot of code can be reused – Dewfy Jul 22 '11 at 12:27
Honestly, when I saw that code I feel like I am too stupid to live... >.> It is so ugly I felt the need to vomit. I never really liked C++... That's why I have chosen C# primarily - it's much more organized and readable (Well, that too depends on the developer). – Vercas Jul 22 '11 at 12:48
@Vercas - actually it is C not C++ – Dewfy Jul 22 '11 at 13:07
Oops - missed that. Now it's even worse. I will accept this answer as it has the highest possibility of helping others, but not me. I will stick to glitches... – Vercas Jul 22 '11 at 13:17
1

@Vercas: don't worry, most people has the vomit feeling at first. If you care about this project, you would probably first reimplement your achive format it in C++. Be aware: if you haven't ever developed in C++, doing it properly is a matter of weeks if not months. – ceztko Jul 22 '11 at 18:11
@ceztko my biggest concern is the streams. My library operates on streams! – Vercas Jul 23 '11 at 09:39

score 1 · Answer 2 · answered Jul 22 '11 at 11:46

1

Yes, it's possible, but you do need Visual C++. File systems are drivers, and they are run natively (in the kernel). The relevant part which you'll need is the Installable File System (IFS) SDK. This is supported on XP, Vista and 7.

Driver programming is a lot harder than normal programming. You have to be a lot more careful with pointers, as they come in different varieties (paged/nonpaged, user/kernel, etc).

answered Jul 22 '11 at 11:46

MSalters

173,980
10
155
350

3

Well, the final battle between me and pointers must take place one day... Maybe today. – Vercas Jul 22 '11 at 11:52
C++ won't help. Drivers are written in plain C and compiled with C compiler that comes with Windows Development Kit (formerly DDK). – Eugene Mayevski 'Callback Jul 22 '11 at 12:10
Well, the product is named "Microsoft Visual C++", and you can certainly use the newer compilers if you know what you're doing, as well as C++ features. See http://msdn.microsoft.com/en-us/windows/hardware/gg487420 for details. – MSalters Jul 22 '11 at 12:16
@MSalters I guess you've read http://msdn.microsoft.com/en-us/windows/hardware/gg487420#EFE, haven't you? That's why C is used, not C++. – Eugene Mayevski 'Callback Jul 22 '11 at 12:19
I even read the entire article, including the summary: "The use of the C++ compiler as a “super-C” is typically expected to work" – MSalters Jul 22 '11 at 12:22
@MSalters and you end up using C++ compiler with plain C code. This is based on 6 years of experience with writing kernel-mode drivers for the above task ;). – Eugene Mayevski 'Callback Jul 22 '11 at 12:27
That says more about you than it says about Windows kernel mode, really. Not that you're the only one. I've seen other developers use Visual C++ as a C compiler even for regular applications. – MSalters Jul 22 '11 at 12:30

score 1 · Answer 3 · answered Jul 22 '11 at 12:26

It's a great idea to learn driver development, if you managed to create a full-scale filesystem. You would need Windows Driver Kit . OSR offerings are way more expensive than Callback File System and they require kernel-mode development, so they won't be an option for you.

Note that it takes 6 to 12 months for experienced Windows developer to create and debug a complex kernel-mode driver (such as file system driver or filter driver or similar). So be ready for a long development time.

Also you would have to also rewrite your existing file system code in C, which would add to your time to develop the solution. The alternative is to re-implement something that Callback File System does, i.e. call user-mode code for all business logic, but this task is comparable (in term) to rewriting your code in C.

A note regarding http://www.acc.umu.se/~bosse/ -- the drivers presented there require almost complete rewriting. We once used them as a guide when creating first version of our SolFS OS edition product. But we ended up rewriting the driver completely (and having wasted time on initial implementation). There are also rumors about IP problems with that code, but I only read them in OSR mailing list so I can't comment much on this (OSR has mailing list archives if you are interested in details).

Yes, I have seen the complexity of those drivers. I can say that it's impossible to me to write my own driver. I am gonna do it the lazy man's way. Dokan; Autostart and Automount; Hidden file... And UAC bypass. — Vercas, Jul 22 '11 at 12:46
Please don't mind... But I want to know - did anyone actually buy or even try your Callback File System? I wanted to *finally* give it a try but I only have a "free" e-mail address... — Vercas, Jul 22 '11 at 13:24
@Vercas Send me an email to cto@eldos.com, and I'll send you a license key. As for "who tried" - we have about a hundred of customers including several very large and very well known companies. Also some Dokan users migrate or already migrated to CBFS. — Eugene Mayevski 'Callback, Jul 22 '11 at 13:55
I know myself a migrating user - La Cie. Wuala is commercial too. <_> — Vercas, Jul 22 '11 at 16:50

Windows File Systems?

3 Answers3