24

on nodejs documentation, the streams section says I can do fs.createReadStream(url || path). But, when I actually do that It tells me Error: ENOENT: no such file or directory. I just want to pipe the video from a readable to a writable stream, But I'm stuck on creating a readable one.

my code:

const express = require('express')
const fs = require('fs')
const url = 'https://www.example.com/path/to/mp4Video.mp4'
const port = 3000

app.get('/video', (req, res) => {
    const readable = fs.createReadStream(url)
})
app.listen(port, () => {
    console.log('listening on port ' + port)
})

the ERROR:

listening on port 3000
events.js:291
      throw er; // Unhandled 'error' event
      ^

Error: ENOENT: no such file or directory, open 'https://www.example.com/path/to/mp4Video.mp4'
Emitted 'error' event on ReadStream instance at:
    at internal/fs/streams.js:136:12
    at FSReqCallback.oncomplete (fs.js:156:23) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: 'https://www.example.com/path/to/mp4Video.mp4'
}

PS: https://www.example.com/path/to/mp4Video.mp4 IS NOT THE ACTUAL URL

Martin Wittick
  • 433
  • 1
  • 4
  • 14

2 Answers2

32

fs.createReadStream() does not work with http URLs only file:// URLs or filename paths. Unfortunately, this is not described in the fs doc, but if you look at the source code for fs.createReadStream() and follow what it calls you can find that it ends up calling fileURULtoPath(url) which will throw if it's not a file: URL.

function fileURLToPath(path) {
  if (typeof path === 'string')
    path = new URL(path);
  else if (!isURLInstance(path))
    throw new ERR_INVALID_ARG_TYPE('path', ['string', 'URL'], path);
  if (path.protocol !== 'file:')
    throw new ERR_INVALID_URL_SCHEME('file');
  return isWindows ? getPathFromURLWin32(path) : getPathFromURLPosix(path);
}

It would suggest using the got() library to get yourself a readstream from a URL:

const got = require('got');
const mp4Url = 'https://www.example.com/path/to/mp4Video.mp4';

app.get('/video', (req, res) => {
    got.stream(mp4Url).pipe(res);
});

More examples described in this article: How to stream file downloads in Nodejs with Got.

You can also use the plain http/https modules to get the readstream, but I find got() to be generally useful at a higher level for lots of http request things so that's what I use. But, here's code with the https module.

const https = require('https');
const mp4Url = 'https://www.example.com/path/to/mp4Video.mp4';

app.get("/", (req, res) => {
    https.get(mp4Url, (stream) => {
        stream.pipe(res);
    });
});

More advanced error handling could be added to both cases.

jfriend00
  • 683,504
  • 96
  • 985
  • 979
  • 2
    The require('https') version worked very well without having to install another 3rd party package. – Martin Cup May 03 '21 at 11:45
  • Does `https.get` return the stream? So can I do something like `const myStream = https.get(mp4Url)`? I'd like to eventually use this with `fluent-ffmpeg`. – Optymystyc May 14 '21 at 15:06
  • @Optymystyc - As you can see in the last code block in my answer, it doesn't return the stream, it provides the stream to the callback you pass it. – jfriend00 May 14 '21 at 16:33
  • Don't forget to set your content type and also catch finish and error events. – JCutting8 Apr 03 '22 at 04:28
  • How do you go about handling a 302 redirect response? Stream doesn't seem to have a status code – JCutting8 Apr 03 '22 at 04:59
  • @JCutting8 - Please write your own question, show your own code and describe your own specific circumstance. In this code, a 302 response from the `https.get()` would just get passed through to the response because `https.get()` does not automatically follow 302 responses. – jfriend00 Apr 03 '22 at 05:04
0

A More Modern Solution that works in Node 18+ is to use fetch which returns a readable stream in response.body

import { Readable } from 'stream';
const readableStream = await fetch('https://www.example.com/path/to/mp4Video.mp4').then(r => Readable.fromWeb(r.body));
readableStream.pipe(...)
Eric Uldall
  • 2,282
  • 1
  • 17
  • 30