0

I am designing a file system in user space and need to test it. I do not want to use the available benchmarking tools as my requirements are different. So to test the file system I wish to simulate file access operation. To do this, I first use the ftw() function to walk through one f my existing file system(experimental) and list all the files and directories in a file.

Then I invoke a simulator to simulate file access by a number of processes. Thus, the simulator randomly starts a process i.e it forks a thread which does what a real process would have done. The thread randomly selects a file operation (read, write, rename etc) selects arguments to this operation from the list(generated by ftw()) . The thread does a number of such file operations and then exits marking the end of a process. The simulator continues to spawn threads; thread execution can overlap just as real processes do. Now, as operations are performed by threads, files get inserted, deleted, renamed and this is updated in the list of files.

I have not yet started coding. Does the plan seem sane? I am also not sure how to code the simulator...how will it spawn threads over a period of time. Should I be using some random delay to do this.

Thanks

Lipika Deka
  • 3,774
  • 6
  • 43
  • 56

2 Answers2

0

Yep, that seems fairly reasonable to me. I would consider attempting to impose a statistical distribution over your file operations (and accesses to particular files) that is somehow matched to your expected workload. You might be able to find some statistics about typical filesystem workloads as a starting point.

Gian
  • 13,735
  • 44
  • 51
  • @Gian..Thanks. I was hinking in similar lines but not sure i know how to do it..Is there some reference I would read up – Lipika Deka Sep 01 '11 at 17:38
  • 1
    http://www.debian-administration.org/articles/388 seems to talk about a workload they used to compare filesystems. Things like copying and then recopying a tree of files would seem like a nice test, as they suggest there. – Gian Sep 01 '11 at 17:43
0

That sounds about right for a decent test case just to make sure it's working. You could use sleep() to wait between spawning threads or just spawn them all at once and have them do an operation then wait a bit, then do another operation, etc... IMO if you hit it hard with a lot of requests and it works then there's a likely chance your filesystem will do just fine. Take an example from PostMark which all it does is append like crazy to different files and other benchmarks that do random access reads/writes in different locations to make sure that the page has to be read from disk.

Jesus Ramos
  • 22,940
  • 10
  • 58
  • 88
  • thanks.. my requirement is to induce inter process file sharing – Lipika Deka Sep 01 '11 at 17:36
  • Actually IPC with files *should* be done by the OS and shared memory pages. If you mean by having files be opened and edited by multiple processes at once then you may not get correctness as the OS (unless you rewrote some stuff) can and will rearrange Block I/O vectors and merge them which will produce incorrect output. – Jesus Ramos Sep 01 '11 at 17:38
  • I have written code to take care of sharing...and that is what i wish to check – Lipika Deka Sep 01 '11 at 17:42