confusions about how images are transferred via HTTP and handled in JavaScript

Question

I learned that the body of a response of an HTTP message can be an arbitrary sequence of bytes so it doesn't need to be text/string. I guess that means when we need to transfer any binary files e.g. images / audio files over HTTP and we don't necessarily need to encode them into text/string using methods like base64, instead, we can just transfer them directly (of course with some compression e.g. gzip)

My first question is, I have not encountered any situations where I need to get actual images or other files via an API call e.g. a REST API call, instead normally we just get the URL to where the image is hosted and we use that URL to display the image in the web page and that seems to be more efficient to me. So I wonder in what circumstances we would have to transfer the actual image file over HTTP?

My second question is, if we are transferring the actual binary data over HTTP, I was looking at this and this and I found that there seems to be two ways to handle the response: 1. read the response as blob by response.blob() 2. read the response as arraybuffer by response.arrayBuffer(). I don't know what the differences are here. What determines whether or not we need to use blob or arraybuffer? Please refer to the below code snippet.

fetch(request, options).then((response) => {
  response.arrayBuffer().then((buffer) => {
  });
});
//  differences?
fetch(request, options)..then(response => response.blob())
  .then(images => {
})

My last question is, after we get the binary data either via response.blob() or response.arrayBuffer(), it seems like we still need to convert the binary data into string using window.btoa i.e. base64 encoding and turn that into URL and then assign it to a img element's src attribute. I guess I don't understand why we would still have to convert the binary data into URL when we already have the data. Is that the only way to display an image in JavaScript, i.e. using src attribute on img tag? Also, according this this and this , I found there seems to be 2 ways to get the URL to a file: 1. converting the buffer to string using btoa and then use data: URL with that string 2. Use URL.createObjectURL directly. Again, I wonder what the differences are here about these two different methods.

Answer 1

For Question 1: When you point the src of an img to a URL in a browser, it will do a 'raw' image GET using HTTP. The content type will be the MIME type of the file and the body will be binary in the file's format. Since MIME is used for SMTP which limits the transfer encoding to 7-bit, many MIME formats are converted to base64 for its portability across many message handlers.

HTTP headers and most content encodings are built over ASCII, a 7-bit subset of text that works with SMTP (email); however, the body of an HTTP response can be pure binary. The browser or the XHR (XmHttpRequest) provides an HTTP client that understands and processes the HTTP headers to give access to the body. It is up to the content producer to ensure the binary matches the format the consumer needs, and it doesn't have to conform to a known file type.

In most cases, the direct transfer without changing the content-type works better, and there are standards so any browser knows how to process a common file. MIME types have expanded to cover all publicly known files but we can always send our own format or user application/octet for binary. In some cases however, when we want error handling or we need to send an 'envelope' of data without an existing file type, we can contain it in an encoding like JSON/XML or multipart form data.

For Question 2: The difference between arrayBuffer and blob have been defined in the referenced answer. For the purpose of reading bytes, they are very similar except that blob is read-only.

For Question 3: The browser can already read base64 encoded binary in an image, so it is a simple way to set the content. It is not converting the binary into a string as much as converting base2 to base64 within an ASCII text readable space--base64 is more of a compression than conversion.

If this seems excessive, it's because it is. If you don't need to download and render images synchronously, don't. Its much more effective to assign the link to the img and let them load in parallel then to parse a huge object and assign its data in series. That level of complexity is only needed (if at all) for saving uploaded data that needs to be processed as a unit.

Answer 2

Answer: Question #1

An example of needing the raw data of an image/video file is actually editing that file. There are lots of libraries that can handle this for you like web-based versions of FFmpeg. Personally I rarely edit raw data without help of an external library, but I have done it before when its necessary. I was trying to "fake" a WebM/MKV file's duration.

Here is the process:

Step 1. Upload a file through HTML's <input type='file'> or through JavaScript's showOpenFilePicker()
Step 2. Read that file as an ArrayBuffer then convert it to a Uint8Array
Step 3. Change the values needed according to the Matroska's Element Specification
Step 4. Create an ArrayBuffer from the edited Uint8Array
Step 5. Create a new File from the ArrayBuffer and save it

Answer: Question #2

The method fetch returns a response. With the response, you can do a variety of things:

• Response.prototype.text()
• Response.prototype.blob()
• Response.prototype.arrayBuffer()
• Response.prototype.json()
• Response.prototype.formData()
  
  

There are many more things to do with a response, but those are the interface methods. A way to think of an arrayBuffer is a Blob's contents. This can be turned into all sorts of other ways to view a file's data.

Answer: Question #3

If you wan't to access the text use Response.prototype.text(). For example with async:

(async function() {
    let res = await fetch('https://www.example.com');
    let text = await res.text();
    //Do whatever with the variable text
})();

or with Promise.prototype.then

fetch('https://www.example.com').then(res=>res.text()).then(text=>{
    //Do whatever with the variable text
});

or turning the raw data into text with TextEncoder

(async function() {
    let res = await fetch('https://www.example.com');
    let buffer = await res.arrayBuffer();
    let uint = new Uint8Array(buffer);
    let decoder = new TextDecoder();
    let decodedText = decoder.decode(uint);
    //decodedText is equal to res.text()
})()