REST API - file (ie images) processing - best practices

Question

We are developing server with REST API, which accepts and responses with JSON. The problem is, if you need to upload images from client to server.

Note: and also I am talking about a use-case where the entity (user) can have multiple files (carPhoto, licensePhoto) and also have other properties (name, email...), but when you create new user, you don't send these images, they are added after the registration process.

---

The solutions I am aware of, but each of them have some flaws

1. Use multipart/form-data instead of JSON

good : POST and PUT requests are as RESTful as possible, they can contain text inputs together with file.

cons : It is not JSON anymore, which is much easier to test, debug etc. compare to multipart/form-data

2. Allow to update separate files

POST request for creating new user does not allow to add images (which is ok in our use-case how I said at beginning), uploading pictures is done by PUT request as multipart/form-data to for example /users/4/carPhoto

good : Everything (except the file uploading itself) remains in JSON, it is easy to test and debug (you can log complete JSON requests without being afraid of their length)

cons : It is not intuitive, you cant POST or PUT all variables of entity at once and also this address /users/4/carPhoto can be considered more as a collection (standard use-case for REST API looks like this /users/4/shipments). Usually you cant (and dont want to) GET/PUT each variable of entity, for example users/4/name . You can get name with GET and change it with PUT at users/4. If there is something after the id, it is usually another collection, like users/4/reviews

3. Use Base64

Send it as JSON but encode files with Base64.

good : Same as first solution, it is as RESTful service as possible.

cons : Once again, testing and debugging is a lot worse (the body can have megabytes of data), there is increase in size and also in processing time in both - client and server

---

I would really like to use solution no. 2, but it has its cons... Anyone can give me a better insight of "what is best" solution?

My goal is to have RESTful services with as much standards included as possible, while I want to keep it as simple as possible.

Answer 1

OP here (I am answering this question after two years, the post made by Daniel Cerecedo was not bad at a time, but the web services are developing very fast)

After three years of full-time software development (with focus also on software architecture, project management and microservice architecture) I definitely choose the second way (but with one general endpoint) as the best one.

If you have a special endpoint for images, it gives you much more power over handling those images.

We have the same REST API (Node.js) for both - mobile apps (iOS/android) and frontend (using React). This is 2017, therefore you don't want to store images locally, you want to upload them to some cloud storage (Google cloud, s3, cloudinary, ...), therefore you want some general handling over them.

Our typical flow is, that as soon as you select an image, it starts uploading on background (usually POST on /images endpoint), returning you the ID after uploading. This is really user-friendly, because user choose an image and then typically proceed with some other fields (i.e. address, name, ...), therefore when he hits "send" button, the image is usually already uploaded. He does not wait and watching the screen saying "uploading...".

The same goes for getting images. Especially thanks to mobile phones and limited mobile data, you don't want to send original images, you want to send resized images, so they do not take that much bandwidth (and to make your mobile apps faster, you often don't want to resize it at all, you want the image that fits perfectly into your view). For this reason, good apps are using something like cloudinary (or we do have our own image server for resizing).

Also, if the data are not private, then you send back to app/frontend just URL and it downloads it from cloud storage directly, which is huge saving of bandwidth and processing time for your server. In our bigger apps there are a lot of terabytes downloaded every month, you don't want to handle that directly on each of your REST API server, which is focused on CRUD operation. You want to handle that at one place (our Imageserver, which have caching etc.) or let cloud services handle all of it.

small 2023 update: If possible, but CDN in front of the pictures, it usually will save you a lot of money and make the pictures even more available (i.e. no issues when peaks happen).

---

Cons : The only "cons" which you should think of is "not assigned images". User select images and continue with filling other fields, but then he says "nah" and turn off the app or tab, but meanwhile you successfully uploaded the image. This means you have uploaded an image which is not assigned anywhere.

There are several ways of handling this. The most easiest one is "I don't care", which is a relevant one, if this is not happening very often or you even have desire to store every image user send you (for any reason) and you don't want any deletion.

Another one is easy too - you have CRON and i.e. every week and you delete all unassigned images older than one week.

Answer 2

There are several decisions to make:

1. The first about resource path:

   • Model the image as a resource on its own:

     • Nested in user (/user/:id/image): the relationship between the user and the image is made implicitly

     • In the root path (/image):

       • The client is held responsible for establishing the relationship between the image and the user, or;

       • If a security context is being provided with the POST request used to create an image, the server can implicitly establish a relationship between the authenticated user and the image.

   • Embed the image as part of the user

2. The second decision is about how to represent the image resource:

   • As Base 64 encoded JSON payload
   • As a multipart payload

This would be my decision track:

• I usually favor design over performance unless there is a strong case for it. It makes the system more maintainable and can be more easily understood by integrators.
• So my first thought is to go for a Base64 representation of the image resource because it lets you keep everything JSON. If you chose this option you can model the resource path as you like.
  • If the relationship between user and image is 1 to 1 I'd favor to model the image as an attribute specially if both data sets are updated at the same time. In any other case you can freely choose to model the image either as an attribute, updating the it via PUT or PATCH, or as a separate resource.
• If you choose multipart payload I'd feel compelled to model the image as a resource on is own, so that other resources, in our case, the user resource, is not impacted by the decision of using a binary representation for the image.

Then comes the question: Is there any performance impact about choosing base64 vs multipart?. We could think that exchanging data in multipart format should be more efficient. But this article shows how little do both representations differ in terms of size.

My choice Base64:

• Consistent design decision
• Negligible performance impact
• As browsers understand data URIs (base64 encoded images), there is no need to transform these if the client is a browser
• I won't cast a vote on whether to have it as an attribute or standalone resource, it depends on your problem domain (which I don't know) and your personal preference.

Answer 3

Your second solution is probably the most correct. You should use the HTTP spec and mimetypes the way they were intended and upload the file via multipart/form-data. As far as handling the relationships, I'd use this process (keeping in mind I know zero about your assumptions or system design):

1. POST to /users to create the user entity.
2. POST the image to /images, making sure to return a Location header to where the image can be retrieved per the HTTP spec.
3. PATCH to /users/carPhoto and assign it the ID of the photo given in the Location header of step 2.

Answer 4

There's no easy solution. Each way has their pros and cons . But the canonical way is using the first option: multipart/form-data. As W3 recommendation guide says

The content type "multipart/form-data" should be used for submitting forms that contain files, non-ASCII data, and binary data.

We aren't sending forms,really, but the implicit principle still applies. Using base64 as a binary representation, is incorrect because you're using the incorrect tool for accomplish your goal, in other hand, the second option forces your API clients to do more job in order to consume your API service. You should do the hard work in the server side in order to supply an easy-to-consume API. The first option is not easy to debug, but when you do it, it probably never changes.

Using multipart/form-data you're sticked with the REST/http philosophy. You can view an answer to similar question here.

Another option if mixing the alternatives, you can use multipart/form-data but instead of send every value separate, you can send a value named payload with the json payload inside it. (I tried this approach using ASP.NET WebAPI 2 and works fine).