According to this msdn article one of the sources for InitialDirectory property used in FileDialog is:
A path that was previously used in the program, perhaps retained from the last directory or file operation.
...
So if you selected your first file from folder x, the next time you try to select a file it will open up the FileDialog with with folder x selected (saving you having to navigate there).
Playing around with notepad this seems to carry across opening a file, saving a file, opening a file and even when printing with "Microsoft XPS Document Writer" which brings up it's own dialog.
So my question is where is this value stored between dialogs? I would like to be able to see what it is and potentially change it? The specific area i would like to change it is in the "Microsoft XPS Document Writer" printer which brings up it's own dialog. So it's not as simple as just setting the initalDirectory Value.