1

In my project, I am assigned to develop a code that can handle large documents to upload. Documents can be pdf, image, etc I need to get this file and save it in the database The client will be React and react will be sending the data as chunks. Backend code is written in Java with Spring MVC. Currently, it is not sending it as a chunk and this is causing memory issues in the Browser.

The basic scenario is the user will upload a large file for ex:200 MB and the front end will be sending it as small packets to backend ie, as chunks.

Currently whatever we have is ,

@RequestMapping(value = "/intake/{channel}/attachment", method = RequestMethod.POST)
    @ResponseBody
    public JsonResponse uploadAttachment(final MultipartHttpServletRequest request,
            @PathVariable(name = "xxxx") String xxxxx, final HttpSession session,
            @RequestParam(name = "xxxxx", required = false, defaultValue = "") String xxxxx,
            @RequestParam(name = "xxxxxx", required = false, defaultValue = "") String xxxxxx,
            @RequestParam(name = "unzip", required = false, defaultValue = "true") boolean unzip,
            @RequestParam(name = "xxx",required = false) String xxx) {

    JsonResponse jsonResponse = new JsonResponse();
    List<String> docsetIds = new ArrayList<>();
    List<String> parentDocsetIds = new ArrayList<>();
    WorklistActionResponse response = null;
    boolean isLiterature = false;

    try {
        List<MultipartFile> files = request.getFiles("attachment");
        if (files != null) {
            String rootURL = CommonUtils.getRootURL(request);
            for (Iterator iterator = files.iterator(); iterator.hasNext();) {
                MultipartFile multipartFile = (MultipartFile) iterator.next();

                FileContent fileContent = new FileContent();
                    fileContent.setContentLength(multipartFile.getSize());
                    fileContent.setContentType(multipartFile.getContentType());
                    fileContent.setInputStream(multipartFile.getInputStream());
                    fileContent.setName(multipartFile.getOriginalFilename());
                    String filePath = multipartFile.getOriginalFilename();
                    String filePathParts[] = filePath.split("/");
                    String fileName = filePathParts[filePathParts.length - 1];
                    String fileParts[] = fileName.split("\\.");
                    String fileExtension = fileParts[fileParts.length - 1];
                    fileContent.setFileExtention(fileExtension);
                    DocumentDetail subDocumentDetail = documentService.saveAttachment(
                            new NewAttachmentRequest(channel, groupId, rootURL, docsetId, fileContent), userId);
                    if (subDocumentDetail == null) {
                        throw new NotSupportedException();
                    }
                    if (subDocumentDetail != null) {
                        subDocumentDetails.add(subDocumentDetail);
                    }
                }
            }
       }
  }

Here the document will be received by the Multipart request. This code is not serving the purpose

The questions I have is :

1, Is multipart request enough to handle chunked data as well.

2, What will be the modification/change I have to make when data comes in Chunk ie, small packets of data will keep on coming until the upload is complete.

I found some similar answers in Stack Overflow fileupload using Chunks and MultipartFileupload

difference between multipart and chunked protocol

How do I receive a file upload in spring mvc using both multipart/form and chunked encoding?

But I can see a lot of contradictory statements to these in Stack Overflow itself.

How can I fix this issue ?

Alex
  • 33
  • 1
  • 6
  • as per my knowledge `MultipartHttpServletRequest` does not have a readymade capability to handle this kind of request, you need to code this yourself by calculating size and end of file detection, either by extending the classes or custom input stream code – Dickens A S Apr 06 '20 at 05:40
  • on the contrary you have workaround with websocket, if your react is really capable of splitting the 200MB file into pieces of Blob you can send it via websocket and combine it at server using a custom code, you can efficiently show a progress bar in front end – Dickens A S Apr 06 '20 at 06:03
  • @DickensAS A S - Can you please elaborate your first comment. – Alex Apr 06 '20 at 08:16
  • HI, as default the facility offered by multi-part upload and its objective is different from your expectation, `Transfer-Encoding: chunked` for a HTTP request is not well defined and supported by `MultipartHttpServletRequest` – Dickens A S Apr 06 '20 at 08:36
  • Hi @DickensAS so how can i achieve this ? – Alex Apr 07 '20 at 04:34
  • Disable spring multipart and use `Apache Commons File Upload API` -- check accepted answer here -- https://stackoverflow.com/questions/32782026/springboot-large-streaming-file-upload-using-apache-commons-fileupload , or use streaming methods – Dickens A S Apr 07 '20 at 05:05

0 Answers0