@Duplicate - this is not a duplicate - the similar question specifically asks about Liferay SendFile method which does not work for larges files. The below question is in regards to simply writing a large file to a Tomcat outpustream. The other question is not answered anyways, rather, they tell him to file an issue with Liferay which is not an answer to any question. The below code is the code for the Liferay SendFile() method. Directing somebody to Liferay support is not a solution.
I am constantly getting an OutOfMemory: Java heap space error when I try and write large files to the HttpServletResponse getOutputStream() - despite attempts at using a buffer, Apache Commons IOUtils.ChunkedOutputStream, writing the output in chunks - nothing works. Same error happened when I use the HttpServletResponse getWriter() method and try it that way also.. But here is my code for the getOutputStream() -
I am writing a download-file feature for my Portlet, and I need it to be able to handle very large files - but it's struggling right now with even 1.4gb files.
final HttpServletResponse httpResponse = PortalUtil.getHttpServletResponse(response);
// Regular Output Stream
BufferedOutputStream outputStream = new BufferedOutputStream(httpResponse.getOutputStream());
// Also tried using this from Apache Commons, still did not work
//ChunkedOutputStream outputStream = new ChunkedOutputStream(outputStream);
// Desired File Download
InputStream is = new FileInputStream(source_file);
while (remainingLength > 0) {
int readBytes = is.read(bytes, 0, (int)Math.min(remainingLength, bufferSize));
if (readBytes == -1) {
break;
}
// Write it
outputStream.write(bytes, 0, readBytes);
// Other Attempts - none of these worked
//IOUtils.writeChunked(bytes, outputStream); //This does not work
//IOUtils.write(bytes, outputStream); // Neither does this
remainingLength -= readBytes;
// Attempted to flush after every 1mb written - but it does not fix the issue
if(buffer >= bufferSize) {
outputStream.flush();
buffer = 0;
} else {
buffer += readBytes;
}
}
// ALTERNATIVELY - We also tried using Apache IOUtils.copy() and copyLarge()
IOUtils.copyLarge(is, outputStream);
//IOUtils.copy(is, outputStream);
// This is in a finally {} block, try{} encloses the above code
IOUtils.closeQuietly(is, outputStream);
The outofmemory exception happens on the outputStream.write() line - and I cannot figure out how to correctly write a file to the output stream that does not cause that exception to get thrown. It works fine for smaller 200mb files, but never for larger ones.
My startup arguments for Tomcat are -Xmx2048m -XX:MaxPermSize=2048m -Xms2048m
Also note: I added a log at the end of the loop to write the total written bytes, and converted it to MB. It gets to 512mb.
The Updated Stack Trace - I restarted the server and got a better one.
The error happens on the outputStream.write() - and if you dig into the stack trace the actual line that throws the error is in UnsyncByteArrayOutputStream.java:91 on the line
byte[] newBuffer = new byte[newBufferSize];
SEVERE: Servlet.service() for servlet filemanager Servlet threw exception java.lang.OutOfMemoryError: Java heap space at com.liferay.portal.kernel.io.unsync.UnsyncByteArrayOutputStream.write(UnsyncByteArrayOutputStream.java:91) at com.liferay.portal.kernel.servlet.ServletOutputStreamAdapter.write(ServletOutputStreamAdapter.java:54) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122) at com.healthmap.FileManager.download(FileManager.java:370) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.liferay.portal.kernel.portlet.LiferayPortlet.callActionMethod(LiferayPortlet.java:163) at com.liferay.util.bridges.mvc.MVCPortlet.callActionMethod(MVCPortlet.java:249) at com.liferay.portal.kernel.portlet.LiferayPortlet.processAction(LiferayPortlet.java:90) at com.liferay.util.bridges.mvc.MVCPortlet.processAction(MVCPortlet.java:212) at com.liferay.portlet.FilterChainImpl.doFilter(FilterChainImpl.java:71) at com.liferay.portal.kernel.portlet.PortletFilterUtil.doFilter(PortletFilterUtil.java:48) at com.liferay.portal.kernel.servlet.PortletServlet.service(PortletServlet.java:112) at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilterChain.doFilter(InvokerFilterChain.java:116) at com.liferay.portal.kernel.servlet.filters.invoker.InvokerFilter.doFilter(InvokerFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:748) at org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:604) at org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:543) at com.liferay.portlet.InvokerPortletImpl.invoke(InvokerPortletImpl.java:583) at com.liferay.portlet.InvokerPortletImpl.invokeAction(InvokerPortletImpl.java:628) at com.liferay.portlet.InvokerPortletImpl.processAction(InvokerPortletImpl.java:308) at com.liferay.portlet.PortletContainerImpl._doProcessAction(PortletContainerImpl.java:389) at com.liferay.portlet.PortletContainerImpl.processAction(PortletContainerImpl.java:107) at com.liferay.portlet.SecurityPortletContainerWrapper.processAction(SecurityPortletContainerWrapper.java:109) at com.liferay.portlet.RestrictPortletContainerWrapper.processAction(RestrictPortletContainerWrapper.java:75)
