Many actions a computer might perform create the need for a counter-balancing "cleanup" action. For example, the act of opening a file creates a need to close it. The act of placing a call with a modem creates the need to disconnect it. Performing an action without performing the required counter-balancing cleanup may sometimes be harmless, but may sometimes have severe consequences in the computer or in the real world (failing to close a file before a user ejects the media may cause file corruption and data loss; failing to terminate a modem connection may cost someone many dollars in excess phone charges).
The purpose of IDisposable is provide a means by which well-behaved programs can make sure that any cleanup actions that need to be performed actually get done. Any routine which performs an operation that must be counter-balanced by cleanup should keep the information and impetus necessary to perform such cleanup until such time as it has been performed; if the routine can't perform such cleanup before returning to the caller, it should either (1) provide the information to the caller, and be documented as requiring the caller to use that information to perform the cleanup, or (2) keep the necessary information in one or more fields, and implement IDisposable; when IDisposable.Dispose is called, the object should use the saved information to perform the cleanup.
Note that IDisposable only works cleanly if any object that requests the creation of an IDisposable object accepts and honors the responsibility of making sure that the object's IDisposable.Dispose method will be called sometime. The system does provide a "backup" means, called finalization, which objects can use to ensure that they get a chance to perform their cleanup. Essentially, an object registers a request with the system that says "Let me know if I've been abandoned". If the system detects that an object has been abandoned, it will try to give the object a chance to perform any necessary cleanup. Note the system may take awhile to notice that an object has been abandoned, and for various reasons the notifications may not always happen. It is almost always much better to call IDisposable.Dispose on objects before they are abandoned, than to rely upon the "last chance" finalization mechanism.