From Wikipedia:
GET - Requests a representation of the specified resource. Requests using GET should only retrieve data and should have no other effect. (This is also true of some other HTTP methods.)[1] The W3C has published guidance principles on this distinction, saying, "Web application design should be informed by the above principles, but also by the relevant limitations."[11] See safe methods below.
POST - Submits data to be processed (e.g., from an HTML form) to the identified resource. The data is included in the body of the request. This may result in the creation of a new resource or the updates of existing resources or both.
PUT - Uploads a representation of the specified resource.
DELETE - Deletes the specified resource.
This, however, doesn't really give you a feel for when to use PUT and DELETE. I found that one of the best ways to get my head round what they do was to tinker with CouchDB. If you're not familiar with it, it's a non-relational database that you access via HTTP.
In CouchDB, you use GET to fetch data, POST to update data, PUT to insert data, and DELETE to delete data. If you'd like to explore it, there's a very good book about it at http://guide.couchdb.org/draft/. Going through http://guide.couchdb.org/draft/tour.html and http://guide.couchdb.org/draft/api.html may give you a good idea of how it works.