0

Following are the HttpServletResponse header and content type used to send a file. Upto 32MB of data is passed to the client, but my whole data is of 100MB size.

response.setContentType("application/octet-stream");
response.setHeader("Transfer-Encoding", "chunked");
response.setHeader("Content-Disposition", "attachment; filename="filename.bin");

I can't store this data to cloud storage as this would be created per user.

Does this can be achieved through HttpServletResponse? or I need to move to some other method, say sockets etc.

Please recommend a method for multi client environment.

jarmod
  • 71,565
  • 16
  • 115
  • 122
srv
  • 433
  • 9
  • 26
  • Hi @Brainislav I am using google app engine to achieve this. GAE tags are required to get visibility. – srv Apr 11 '16 at 13:04
  • Does that (32MB problem) happen with any browser, or just with particular one. And it might be useful to set the `Content-Length` header. See also http://stackoverflow.com/questions/685271/using-servletoutputstream-to-write-very-large-files-in-a-java-servlet-without-me – Jozef Chocholacek Apr 11 '16 at 19:47
  • You should set chunked transfer mode via the API, not this way. It won't actually do any chunking unless you tell it to via the API. – user207421 Apr 11 '16 at 20:54
  • @EJP How to do that? Please share a reference. – srv Apr 11 '16 at 21:11
  • Err, oops, there isn't an API for that in `HttpServletResponse`, but that's even more of a reason not to set the header. There won't be any chunked encoding unless the servlet container does it, in which case it will insert its own chunking and its own header. In Tomcat this is done by setting a [`Connector` property](https://tomcat.apache.org/tomcat-4.0-doc/config/http11.html). – user207421 Apr 11 '16 at 23:11

2 Answers2

1

App Engine has quotas, designed to protect everyone from runaway applications. The maximum amount of data your application can receive in a single inbound HTTP request is 32MB.

If you want to upload larger files then you should look at Blobstore API.

See the quota documentation for more details.

jarmod
  • 71,565
  • 16
  • 115
  • 122
  • Hi @jarmod, I got your point. I want to download generated data, is similar Blobstore/Cloud Storage shall work? Please share if you have an idea. – srv Apr 12 '16 at 15:02
  • If you want to upload, or download, large files then I think you should probably use GCS and your client should be interacting directly with the GCS service for upload/download. In the case of downloads, your app server can create a time-limited, signed URL, provide it to the client, and the client can then securely download the file directly from GCS. You should avoid making your app server behave as a proxy server for uploads to, and downloads from, GCS. – jarmod Apr 12 '16 at 18:13
1

You cannot get around the 32mb limit on requests. Your best bet is to push the data to GCS, and generate a signed URL.

Depending on what you need to do to generate your 100mb file, you may need to offload the whole process using the task queue to prevent request timeouts.

Nick
  • 1,822
  • 10
  • 9