REST: Prevent creation of duplicate resources via conditional POST?

Question

I've been searching for best practices for preventing the accidental creation of duplicate resources when using POST to create a new resource, for the case where the resource is to be named by the server and hence PUT can't be used. The API I'm building will be used by mobile clients, and the situation I'm concerned about is when the client gets disconnected after submitting the POST request but before getting the response. I found this question, but there was no mention of using a conditional POST, hence my question.

Is doing a conditional POST to the parent resource, analogous to using a conditional PUT to modify a resource, a reasonable solution to this problem? If not, why not?

The client/server interaction would be just like with a conditional PUT:

1. Client GETs the parent resource, including the ETag reflecting its current state (which would include its subordinate resources),

2. Client does a conditional POST to the parent resource (includes the parent's ETag value in an If-Match header) to create a new resource,

3. Client gets disconnected before getting the server response, so doesn't know if it succeeded,

4. Later, when reconnected, the client resubmits the same conditional POST request,

5. Either the earlier request didn't reach the server, so the server creates the resource and replies with a 201, or the earlier request did reach the server, so the server replies with a 412 and the duplicate resource isn't created.

Answer 1

Your solution is clever, but less than ideal. Your client may never get his 201 confirmation, and will have to interpret the 412 error as success.

REST afficianados often suggest you create the resource with an empty POST, then, once the client has the id of the newly created resource, he can do an "idempotent" update to fill it. This is nice, but you will likely need to make DB columns nullable that wouldn't otherwise be, and your updates are only idempotent if no-one else is trying to update at the same time.

According to ME, HTTP is flaky. Requests timeout, browser windows get closed, connections get reset, trains go into tunnels with mobile users aboard. There's a simple, robust pattern for dealing with this. Unsafe actions should always be uniquely identified, and servers should store, and be able to repeat if necessary, the response to any unsafe request. This is not HTTP caching, where a request may be served from cache but the cache may be flushed for whatever reason. This is a guarantee by the server application that if an "action" request is seen a second time, the stored response will be repeated without anything else happening. If the action identity is to be generated by the server, then a request-response should be dedicated just to sending the id. If you implement this for one unsafe request, you might as well do it for all of them, and in so doing you will escape numerous thorny problems: successive update requests wiping out other users' changes, or hitting incompatible states ("order already submitted"), successive delete requests generating 404 errors.

I have a little google doc exploring the pattern more fully if you're interested.

Answer 2

I think this scheme would work. If you want to ensure POST does not result in duplicates, you need the client to send something unique in the POST. The server can then verify uniqueness.

You might as well have the client generate a GUID for each request, rather than obtaining this from the server via a GET.

Your steps then become:-

1. Client generates a GUID
2. Client does a POST to the resource, which includes the GUID
3. Client gets disconnected and doesn't know if it succeeded
4. Client connects again and does another POST with the same GUID
5. Server checks the GUID, and either creates the resource (if it never received the first POST) or indicates that this was a duplicate

It might be more restful to use PUT, and have the client decide the resource name. If you did not like the choosen name, you could indicate that you had created the resource but that it's canonical location was somewhere of the server's choosing.

Answer 3

Why not simply do duplicate detection on the server based on the actual resource, using whatever internal mechanism the server chooses to use.

It's just safer that way.

Then you return the URL to the appropriate resource (whether it was freshly created or not).

If the parents ETag is based on the state of sub resources, then it's not a reliable mechanism to check for "duplicate resources". All you know is that the parent has "changed", somehow, since last time. How do you even know it's because your old POST was processed after disconnect? Could be anything changed that ETag.

This is basically a optimistic locking scenario being played out, and it comes down to another question. If the resource is already created, what then? Is that an error? Or a feature? Do you care? Is it bad to send a creation request that's silently ignored by the server when the resource already exists?

And if it already exists, but is "different" enough (i.e. say the name matches but the address is different), is that a duplicate? is that an update? is that a error for trying to change an existing resource?

Another solution is to make two trips. One to stage the request, another to commit it. You can query the status of the request when you come back if it's interrupted. If the commit didn't got through, you can commit it again. If it did, you're happy and can move on.

Just depends on how unstable your comms are and how important this particular operation is whether you want to jump through the hoops to do it safely.