Fetching a list of URLs from an array and processing them

Question

I'm trying to fetch a list of URLs in an array and then covert them all to Base64. I however need to block the function from running until all the images are fetched and processed since the output is required for the next step.

The expected process should be let example = ["https://www.example.com/1.jpg", "https://www.example.com/2.jpg"]; converted to a Base64 encoded array.

//Contains the B64 encoded images
var observationImages = []
//List of Images to fetch and encode
 var lookupPhotos = ['https://www.example.com/1.jpg', 'https://www.example.com/2.jpg'];

    Promise.all(
            lookupPhotos.map(url => {
                fetch(url)
                    .then(response => response.blob())
                    .then(blob => new Promise((resolve, reject) => {
                        console.log('Converting to b64');
                        const reader = new FileReader()
                        reader.onloadend = () => resolve(reader.result)
                        reader.onerror = reject
                    }))
                    .then(dataUrl => {observationImages.push(dataUrl)});
            }
        )
    ).then(data => {
            console.log('Promises all resolved!');
            console.log(observationImages);
    });

I've tried using Promises but I'm not sure I fully understood how they work so it doesn't work as expected.

Possible duplicate of [CONVERT Image url to Base64](https://stackoverflow.com/questions/22172604/convert-image-url-to-base64) — Arsalan Akhtar, Sep 24 '19 at 07:47
I already can convert to Base64 the question is more how to do it synchronously for each image in the array, — David, Sep 24 '19 at 07:55
Hi, can you share your code? As I can see, you can pipeline your promises to ensure that each request call is finished before you process the images — Kenzo-kun, Sep 24 '19 at 08:07
Similar question, but does not include the custom promise that you made. https://stackoverflow.com/questions/58075409/javascript-base64-how-to-fetch-a-list-of-urls-frrom-an-array-and-convert-them/58076203#58076203 — Maiya, Sep 24 '19 at 09:11

Maiya · Answer 1 · 2019-09-25T01:07:09.520

This code is working locally in my browser.

Can you see what I changed?

declared your custom promise function outside of the promise chain, just for clarity
passed the blob to the reader.
put a return in your map callback, so that the promise is being stored to the array (otherwise you just will get an empty array).
resolve and reject are methods that you pass data to. That data gets passed down the promise chain.
The chains that you are storing in the array need to return a promise. So, moved the non-promise code to after the Promise.all() was resolved.

added a .catch at the end.

// will contain the B64 encoded images
let observationImages = [];
//List of Images to fetch and encode
let photosToLookUp = [
"https://www.gravatar.com/avatar/a70aa9d494bf438c8b56aced999e897f?s=48&d=identicon&r=PG",
"https://www.gravatar.com/avatar/e0e123ac05632003d0ff22ec8ce3a554?s=32&d=identicon&r=PG"
];

const customPromise = blob => {
     return new Promise((resolve, reject) => {
          console.log("Converting to b64");
          const reader = new FileReader();
          reader.readAsDataURL(blob);
          reader.onloadend = () => {
              resolve(reader.result);
          };
          reader.onerror = err => {
               reject(err);
          };
      });
  };

Promise.all(
   photosToLookUp.map(url => {
       return fetch(url)
           .then(response => {
               return response.blob();
            })
           .then(blob => {
               return customPromise(blob);
            });
     })
  )
  .then(arrayOfData => {
      console.log("Promises all resolved!");
      arrayOfData.forEach(dataUrl => {
           observationImages.push(dataUrl);
      });
      //there is no return value here because you've gotten all the info from the promise chain and are processing it.
})
.catch(err => {
     console.log("error:", err);
 });

There's a button in the menu bar when you write a post. https://stackoverflow.blog/2014/09/16/introducing-runnable-javascript-css-and-html-code-snippets/ — Maiya, Sep 24 '19 at 19:25
The link you posted is broken. Maybe they should. I can try to edit it when I have time. — Maiya, Sep 24 '19 at 19:40
[How do I create a runnable stack snippet?](https://meta.stackoverflow.com/questions/358992) — Scott Sauyet, Sep 24 '19 at 19:49
It's challenging to make a snippet with this code because he is using fetch() and the sandbox is causing CORS errors. — Maiya, Sep 24 '19 at 20:01
Ah, of course. I know I've done some `fetch` snippets before but that was with something like http://jsonplaceholder.typicode.com/, which probably sends appropriate headers to help. — Scott Sauyet, Sep 24 '19 at 20:24

score 0 · Accepted Answer · answered Sep 24 '19 at 20:46

The biggest issue with your code is that you don't actually use the reader you configure! You need something like

reader .readAsDataURL (blob)

somewhere in your code. But here is a different take on it that tries to break the process down into more discrete steps. I hope it's clear:

const toBase64 = (blob) => 
  new Promise ((resolve, reject) => {
    const reader = new FileReader()
    reader .onload = () => resolve (reader .result .split (',') [1])
    reader .onerror = error => reject (error)
    reader .readAsDataURL (blob)
  })

const getImageAsBase64 = (url) => 
  fetch (url)
    .then (resp => resp .blob ())
    .then (toBase64)

const getAllImagesAsBase64 = (urls) => 
  Promise .all (urls .map (getImageAsBase64))

const urls = [
  'https://www.gravatar.com/avatar/a70aa9d494bf438c8b56aced999e897f?s=48&d=identicon&r=PG',
  'https://www.gravatar.com/avatar/e0e123ac05632003d0ff22ec8ce3a554?s=32&d=identicon&r=PG'
]

getAllImagesAsBase64 (urls)
  .then (console.log)

He mentioned he had the reader part figured out in one of the comments, so I think it might be pseudocode? — Maiya, Sep 24 '19 at 21:40
Well it's definitely broken code; there are a number of compilation errors, some still in your version. That's one of the reasons I like the snippets! Feel free to steal my URLS if you want to edit to include a snippet. — Scott Sauyet, Sep 25 '19 at 00:03
Thanks. It's hard for me to find them when not running the actual code. I'll steal your urls :) — Maiya, Sep 25 '19 at 00:15

Fetching a list of URLs from an array and processing them

2 Answers2

This code is working locally in my browser.