Node Js How to download file into memory without writing the file to the system or without creating the file in a directory

Question

I have a feature in my program where in i download all the files from a server to my directory and then get the path of those file and name and data after being downloaded and store in to mongo database using mongo grid fs.but I have a problem isn't its already redundant that I download the file and then getting the file data ang storing it to mongo db. Is there a way we can download file in node js without writing it to disk or directory but still we could get the file data? so that I can still store those file in mongo without those file being created on my directory like for example storing it to memory ? and get deleted after all was finished.

The reason I want to implement this is that I no longer need to delete the files being downloaded to my directory after I store those files to mongo.

sample code on downloading file.

var download = function (url, dest, callback) {

        request.get(url)
            .on('error', function (err) { console.log(err) })
            .pipe(fs.createWriteStream(dest))
            .on('close', callback);

    };

    final_list.forEach(function (str) {
        var filename = str.split('/').pop();

        console.log('Downloading ' + filename);

        download(str, filename, function () { console.log('Finished Downloading' + "" + filename) });
    });

Code on storing on mongo

 var tempfile = dest;
                var origname = filename;
                var writestream = gfs.createWriteStream({ filename: origname });

                console.log('Finished Downloading' + " " + filename);

                fs.createReadStream(tempfile)
                    .on('end', function () {
                    })
                    .on('error', function () {
                    })
                    .pipe(writestream);

Of course, in the same way you read the file and write it to disk, just leave out the bit where you are going to write it to disk and then buffer it. You will need to create a collection to store your files it which is simple enough. — SPlatten, Sep 27 '19 at 06:21
@SPlatten , can you post an answer Sir ? based on my code above , thank you. — , Sep 27 '19 at 06:22
@SPlatten , can i store the file using its bit ? cause in my current code today I downloaded the file first and then store it to mongo — , Sep 27 '19 at 06:35
@SPlatten, I have no problem on storing or when it comes to mongo , my problem is just on getting the file data without downloading it to disk — , Sep 27 '19 at 06:41
@Mr.MarkTawin, I'm not entirely sure what you are asking, if you have the data then what is the issue? — SPlatten, Sep 28 '19 at 07:04

score 0 · Accepted Answer · answered Sep 27 '19 at 06:44

0

You could just directly pipe the request stream to the gridfs write stream:

 const fileStorage = gfs.createWriteStream({ filename: dest });

 request.get(url)
        .on('error', function (err) { console.log(err) })
        .pipe(fileStorage)
        .on('close', callback);

answered Sep 27 '19 at 06:44

Jonas Wilms

132,000
20
149
151

so ill be removing tr, filename, from the download ( – Sep 27 '19 at 06:57
rather filename you will have to pass the stream – AZ_ Sep 27 '19 at 06:58
Could you please take your time to *research* the code shown and at least partly *understand* it, then ask one question at a time. Please don't spam me with comments. Thank you. And for sure this isn't a finished implementation, it's meant to give you the basic idea – Jonas Wilms Sep 27 '19 at 07:00
@JonasWilms , I have made it work already Sir , thank you for the idea – Sep 27 '19 at 07:09
Glad to help :) – Jonas Wilms Sep 27 '19 at 07:10
@JonasWilms , what do you mean by this sir "rather filename you will have to pass the stream" ? – Sep 27 '19 at 07:26
That wasn't me. – Jonas Wilms Sep 27 '19 at 07:32
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/200051/discussion-between-mr-mark-tawin-and-jonas-wilms). – Sep 27 '19 at 07:32

Node Js How to download file into memory without writing the file to the system or without creating the file in a directory

sample code on downloading file.

Code on storing on mongo

1 Answers1

Linked