Buffering response in heap space cause problems for large files

Question

I have a web server project, i get an exception while trying to download large files. The file is read and written to ServletOutputStream via streams.

Sample code :

private void readFromInput(BufferedInputStream fis,
    ServletOutputStream sout) throws IOException
    {
    byte[] buf = new byte[4096];
    int c = 0;
    while ((c = fis.read(buf)) != -1)
    {
        sout.write(buf, 0, c);    
    }
    fis.close();
}

When i look at the backtrace, i see some filters are executed.

Here is the some parts of the exception :

javax.servlet.ServletException: #{DownloaderBean.actionDownload}: 
java.lang.OutOfMemoryError: Java heap space
javax.faces.webapp.FacesServlet.service(FacesServlet.java:256)
org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:144)
org.ajax4jsf.framework.ajax.xmlfilter.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:127)
org.ajax4jsf.framework.ajax.xmlfilter.BaseFilter.doFilter(BaseFilter.java:277)
....
....
....

java.lang.OutOfMemoryError: Java heap space
java.io.ByteArrayOutputStream.write(Unknown Source)
org.apache.myfaces.webapp.filter.ExtensionsResponseWrapper$MyServletOutputStream.write(ExtensionsResponseWrapper.java:135)

When i look at that ExtensionFilter code :

http://grepcode.com/file/repo1.maven.org/maven2/org.apache.myfaces.tomahawk/tomahawk12/1.1.7/org/apache/myfaces/webapp/filter/ExtensionsFilter.java

There is a part on this page :

"When the ExtensionsFilter is enabled, and the DefaultAddResources implementation is 
used then there is no way to avoid having the response buffered in memory"

I guess these filters buffer the response on heap and cause the problem. Is there a way to prevent this filters apply on a spesific page/link? Or should i follow another way to handle this?

Answer 1

The MyFaces ExtensionsFilter is apparently buffering the entire response in server's memory until the last bit. You've thus basically 2 options:

1. Get rid of the MyFaces ExtensionsFilter.

2. Don't let the request hit the MyFaces ExtensionsFilter.

Option 1 is maybe drastic if you actually need it for some functional requirement in your web application, but doable if an alternative can be found. E.g. if you merely needed it to process file uploads, then you may consider using an alternative component library or even standard JSF 2.2 one for this.

Option 2 is doable in 2 ways:

1. Change the filter's URL pattern so that the download request doesn't hit it. If you can figure out on which URLs exactly you need the ExtensionsFilter, then you could alter its <filter-mapping> accordingly so that it only kicks in on exactly those URLs instead of globally on the FacesServlet.

   E.g. When it should be invoked on /upload.jsf only, replace <servlet-name> by <url-pattern>:

   <filter-mapping>
       <filter-name>MyFacesExtensionsFilter</filter-name>
       <url-pattern>/upload.jsf</url-pattern>
   </filter-mapping>
   

   This is only troublesome when you actually perform the download action from the very same page.

2. Change the download request URL so that it doesn't hit the filter. Provided that you can't just put those files in the public web content, nor can add the folder with the files as another context (e.g. because those files are generated on the fly), one way would be moving all the download serving code from the JSF managed bean to a plain vanilla servlet. Then just let the link URL or form action point to that servlet instead. As that request won't hit the FacesServlet, the ExtensionsFilter won't be hit either.

   E.g.

   @WebServlet("/files/*")
   public class FileServlet extends HttpServlet {
   
       protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
           String filename = request.getPathInfo().substring(1);
   
           // Just do your job to get the File or InputStream, depending on the functional requirements.
           // This kickoff example just allocates a file in the file system.
           File file = new File("/path/to/files", filename);
           response.setHeader("Content-Type", getServletContext().getMimetype(filename));
           response.setHeader("Content-Length", String.valueOf(file.length()));
           response.setHeader("Content-Disposition", "attachment; filename=\"" + filename + "\"");
           Files.copy(file.toPath(), response.getOutputStream());
       }
   
   }
   

   (note: if you're still not on Servlet 3.0 yet, just replace @WebServlet by the usual servlet mapping in web.xml; and if you're still not on Java 7 either, just replace Files#copy() by the usual InputStream/OutputStream loop boilerplate)

   Invoke it like follows (assuming legacy JSF 1.2 on JSP, given the fact that you linked to source code of Tomahawk for JSF 1.2; and thus EL in template text isn't supported).

   <h:outputLink value="#{request.contextPath}/files/#{bean.filename}">
       <h:outputText value="Download #{bean.filename}" />
   </h:outputLink>
   

   If the download requires additional parameters, just pass them using <f:param>:

   <h:outputLink value="#{request.contextPath}/files/#{bean.filename}">
       <f:param name="foo" value="#{bean.foo}" />
       <f:param name="bar" value="#{bean.bar}" />
       <h:outputText value="Download #{bean.filename}" />
   </h:outputLink>
   

   which can then be obtained in the servlet as follows:

   String foo = request.getParameter("foo");
   String bar = request.getParameter("bar");
   // ...
   

Answer 2

Change the download URL so that it doesn't match any mapped URL in your webapp, so that the default Servlet will handle it, without any of these pesky filters. No code required at all.

Or stick an Apache HTTP in front that will serve the file directly and proxy other requests to your Servlet container.

Answer 3

You can solve this issue by excluding download URLs from the extension filter and we can do this by creating a custom extension filter and excluding download URLs as follows:

Steps 1 . create a custom filter as follows

public class CustomExtensionsFilter implements Filter { 


private ExtensionsFilter extensionFilter = new ExtensionsFilter();
private List excludedUrls;

public void doFilter(ServletRequest req, ServletResponse resp, FilterChain chain)
        throws IOException, ServletException {

    String requestURI = ((HttpServletRequest) req).getRequestURI();
    

    if (!excludedUrls.contains(requestURI)) {
        System.out.println(" passing to  ExtensionsFilter  " + requestURI);
        extensionFilter.doFilter(req, resp, chain);
    }

    // Forward the request to the next filter or servlet in the chain.
    chain.doFilter(req, resp);
}

public void init(FilterConfig filterConfig) {

    String excludePattern = filterConfig.getInitParameter("excludedUrls");
    
    excludedUrls = Arrays.asList(excludePattern.split(","));

    extensionFilter.init(filterConfig);
}

public void destroy() {
    extensionFilter.destroy();
}

}

steps 2. replace extension filter class with custom extension filter class

<filter>
    <filter-name>MyFacesExtensionsFilter</filter-name>
    <filter-class>com.filters.CustomExtensionsFilter</filter-class>
    <init-param>
        <param-name>maxFileSize</param-name>
        <param-value>20m</param-value>
    </init-param>
    
    <init-param>
        <param-name>excludedUrls</param-name>
        <param-value>/faces/largefiles/download.xhtml</param-value>
    </init-param>
</filter>

<filter-mapping>
    <filter-name>MyFacesExtensionsFilter</filter-name>
    <servlet-name>faces</servlet-name>
</filter-mapping>

and you can add more urls as comma separated in excludedUrls param