I'm assuming you're looking for an external way to get those dumps created (after all, you're asking about exceptions that are not handled).
In Vista and above, the mechanism you're looking for is called Windows Error Reporting - WER. You can configure it to create dumps upon crash, and place them at a certain location. Prior to Vista, you could do that using drwtsn32.exe. You can also run your program using ADPlus, which will create a dump as soon as the process crashes, without the need for any system-wide configuration.
If you'd like to create a dump from the program itself, you can use MiniDumpWriteDump , but that's less advisable. When in an unclear state, it's better to let some other process take the snapshot of yours.
As for WinDbg, there's really a ton of information on the web. I guess you can start here, but you should ask specific questions if you have specific problems.