Well, for one thing if the "handle exception" part throws an exception itself, cleanup won't occur.
More importantly though, you should almost never be catching all exceptions. You should catch very specific exceptions you can handle, and let other exceptions bubble up. At that point, you must use a finally
block if you want the cleanup to still occur.
It's not clear what language you're using, but if it's Java then there's already a difference, as non-Exception exceptions (other subclasses of Throwable) will end up cleaning up in your first version but not in your second - but you shouldn't even catch Exception
generally.
Personally I find I write more try/finally blocks than try/catch or try/catch/finally blocks. I find it pretty rare that I can really deal with an exception... although sometimes I catch one exception just to convert it to one more appropriate for the abstraction level I'm working on, and then rethrow.
EDIT: As noted in dj aqeel's answer, finally
statements are also executed if the block completes without an exception, e.g. through a return
statement. The very fact that I'd forgotten that is a good reason to favour finally
: it promotes consistency. It's one consistent place to perform clean-up, regardless of how the block is exited.
Also note that in C#, you'd idiomatically use a using
statement for disposable resources. And Java 7 has the try-with-resources statement.