In your second case if the code rethrows an exception or return from the catch block then your cleanup
code will not be called. In case of finally
block it will be executed even if you have an exception or a return statement from the catch block.
The MSDN says:
By using a finally block, you can clean up any resources that are
allocated in a try block, and you can run code even if an exception
occurs in the try block. Typically, the statements of a finally block
run when control leaves a try statement. The transfer of control can
occur as a result of normal execution, of execution of a break,
continue, goto, or return statement, or of propagation of an exception
out of the try statement.