15

I want to have an http method that sends the file to the user, but it needs some time (e.g. 4 seconds) to generate file content.

What I want, is the browser to instantly show the file as being downloaded. But Chrome only shows the file as being downloaded after 8 bytes are send. I don't know the first 8 bytes of my file upfront. Firefox, however, shows the download instantly.

Here's the example (in Express, but backend technology doesn't matter, I had the same example in ASP.Net):

const express = require('express');

const app = express();
const sleep = ms => new Promise(resolve => setTimeout(resolve, ms))

app.get('/:type?', async (req, res) =>  {
  res.set("Content-type", "application/octet-stream");
  res.set("Content-Disposition", "attachment;filename=\"Report.txt\"");

  res.write('1234567'); 
  if (req.params.type == "instant")
    res.write('8'); //if I send 8 bytes before sleep, file downloading appears instantly
  await sleep(4*1000);

  res.write('9');
  res.end();
});

app.listen(3000, () => {
  console.log('server started');
});

https://repl.it/@ArturDrobinskiy/AllJumboSpellchecker?language=nodejs

Is there a way to solve this?

Example URLs with the code above:

Shaddix
  • 5,901
  • 8
  • 45
  • 86
  • Is your server behind a reverse proxy like nginx?? – A_C Oct 24 '18 at 03:38
  • What is the size of your file? – A_C Oct 24 '18 at 03:41
  • I don't know the size of the file upfront, let's say about 300KB. – Shaddix Oct 25 '18 at 07:56
  • I adjusted the links to repl.io - it's reproducible there, without reverse proxies/nginx – Shaddix Oct 25 '18 at 08:21
  • I am kind of confused what you are trying to achieve. If I download the file and check its size, it shows 8 bytes only in both your links and download happens instantly – A_C Oct 25 '18 at 08:48
  • If I am not wrong, you could refer to this:- https://thoughts.t37.net/nginx-optimization-understanding-sendfile-tcp-nodelay-and-tcp-nopush-c55cdd276765 . The property TCP_NODELAY of TCP stack might be of help to you. – A_C Oct 25 '18 at 08:54
  • For more lookup: https://www.extrahop.com/company/blog/2016/tcp-nodelay-nagle-quickack-best-practices/ , https://stackoverflow.com/questions/3761276/when-should-i-use-tcp-nodelay-and-when-tcp-cork – A_C Oct 25 '18 at 08:58
  • thanks for the links, I will review them. The file on the first link is 8 bytes ("12345679"), the second link is 9 bytes ("123456789"). Haven't you noticed a 4 second delay before first file started to download? – Shaddix Oct 25 '18 at 09:03
  • Read your file in chunks, and write each chunk as they are read from memory? And why do you not know the size of your file? Doing streaming like this will also decrease download time. – skalet Oct 25 '18 at 09:16
  • One of the ways of chunking via nginx is : http://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile_max_chunk – A_C Oct 25 '18 at 09:19
  • @skalet you can't always stream chunks as you read them. If you're sending an xls file for example , you will need to fully prepare it in the server and then send it – Tourki Oct 29 '18 at 21:44
  • 1
    @Shaddix did you try the same thing in firefox ? what was the behaviour ? – Tourki Oct 29 '18 at 21:47
  • @Turkish in Firefox both links are instant! Thanks, it doesn't solve the problem, but it's a major detail! – Shaddix Oct 30 '18 at 10:04
  • @Shaddix you could send 8 bytes of empty lines as a workaround, I don't think it would affect your file's content – Tourki Oct 30 '18 at 10:29
  • well, those 8 bytes of newlines will be added to the file itself (I adjusted the repl example) – Shaddix Oct 30 '18 at 11:36

1 Answers1

0

You could try prepending the file with 0-width space characters. 

const express = require('express');

const app = express();
const sleep = ms => new Promise(resolve => setTimeout(resolve, ms))

app.get('/:type?', async (req, res) =>  {
  res.set("Content-type", "application/octet-stream");
  res.set("Content-Disposition", "attachment; filename=\"Report.txt\"");
  res.write('\u200B\u200B\u200B\u200B\u200B\u200B\u200B\u200B'); 
  //res.write('1234567'); 
  if (req.params.type == "instant")
    res.write('8'); //if I send 8 bytes before sleep, file downloading appears instantly
  await sleep(4*1000);

  res.write('9');
  res.end();
});

app.listen(3000, () => {
  console.log('server started');
});
Pika Supports Ukraine
  • 3,612
  • 10
  • 26
  • 42
daz
  • 714
  • 6
  • 9
  • @Shaddix Oh thought it would work since it made it download instantly on firefox which is what I was using. If it works in curl, then the server is probably not the cause of this problem. Would it be fine to prepend 8 U+200B characters in front of the file? U+200B is the zero width space character so it doesn't effect how the file looks. Otherwise, I would try to see if you could stream the file as it's processed instead of waiting for the processing to finish, so that the file starts downloading immediately. I have no idea how this file is generated though. – daz Oct 30 '18 at 20:51