1

I am hosting a NodeJS app on Heroku in which users can upload their own images. I am hosting those images on Google Cloud Storage because Heroku will wipe them if they weren't part of the deployment. So I check to see if the images are on the server. If they aren't on the server, then they are requested from GCS.

When I try to download multiple images from a GCS Bucket to my host server, all image files downloaded will be the same as the last image in the request. If I have 1 image in the request, then the downloaded image will be correct. If I have more than one file, then only the last downloaded image is correct. The rest of the files have the correct filename, but the image downloaded for all of them are identical.

const fs = require('fs');
const path = require('path');
const { Storage } = require('@google-cloud/storage');
const mongoose = require('mongoose');
const User = mongoose.model('Users');
const filePath = 'uploads';

//GET REQUEST WILL START HERE
const recipesReadList = (req, res) => {
  getUser(req, res, (req, res) => {
    User.findById(req.params.userid)
      .select('recipe')
      .exec((err, user) => {
        if (user.recipe && user.recipe.length > 0) {
          user.recipe.forEach((recipe) => {
            if (!fs.existsSync(`uploads/${recipe.imageName}`)) {
              downloadImage(recipe.imageName);
            }
          });
          return res.status(200).json(user.recipe);
        }
      });
  });
};

const getUser = (req, res, callback) => {
  if (req.payload && req.payload.email) {
    User.findOne({ email: req.payload.email }).exec((err, user) => {
      callback(req, res, user.userName);
    });
  }
};

const downloadImage = async (file) => {
  downloadFromGCS(file).catch(console.error);
};

const GCS_Storage = new Storage({
  projectId: process.env.GCLOUD_PROJECT,
  keyFilename: path.join(__dirname, '../../', `${process.env.GCS_KEYFILE}`),
});

let downloadFileName = '';

async function downloadFromGCS(fileName) {
  downloadFileName = fileName;
  let foundFile = false;

  const options = {
    destination: filePath + '/' + fileName,
  };

  const [files] = await GCS_Storage.bucket(process.env.GCS_BUCKET).getFiles();

  files.forEach((file) => {
    if (file.name === fileName) {
      foundFile = true;

      // Download the file to host storage
      GCS_Storage.bucket(process.env.GCS_BUCKET)
        .file(downloadFileName)
        .download(options);
    }
  });

  if (foundFile === false) {
    console.log(`File not found on cloud storage: ${fileName}`);
  }
}

module.exports = {
  recipesReadList,
};

I have tried using async/await (awaiting the download) based on this post, but that yielded the same results. I'm not sure how to validate if a file on the server has completed a download.

JonW
  • 41
  • 6
  • 1
    You're going to have to show more code and explain more clearly what isn't working the way you expect with that code using specific examples. What you have now is incomplete. Please read: https://stackoverflow.com/help/minimal-reproducible-example – Doug Stevenson Dec 30 '21 at 02:51
  • @DougStevenson, I updated my code. In the process, I solved the issue. I was looking in the wrong place. Thanks for the nudge! Sadly, I don't understand WHY it was an issue. – JonW Dec 31 '21 at 21:27

1 Answers1

1

I was creating a reprex code sample, per request from @DougStevenson (thanks for that, I was looking in the wrong place), and I solved the issue. At some point in development, I had created a variable outside the scope of a function because it wasn't behaving as expected. I don't remember exactly what it was doing at the time. Regardless, changing the scope of the variable fixed the issue.

I changed this:

let downloadFileName = '';

async function downloadFromGCS(fileName) {
  downloadFileName = fileName;

to this

//let downloadFileName = '';
    
async function downloadFromGCS(fileName) {
  let downloadFileName = fileName;

And now the issue is resolved.

JonW
  • 41
  • 6