Spring async file upload and processing

Question

I'm trying to upload file and then read it, everything is working fine, but not when I put @Async annotation on the handler method.

I don't want the user to keep waiting until it processes the file. But after putting this annotation I get java.lang.IllegalStateException: File has been moved - cannot be read again exception. What happens and how do I fix this? As I understand, Spring could be just clearing the file because request-response ends and it cleans it up. But shouldn't @Async prevent this?

Sample Spring Boot application:

@SpringBootApplication
@EnableSwagger2
@ComponentScan(value = "hello")
@EnableAsync
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    public Docket api() {
        return new Docket(DocumentationType.SWAGGER_2)
                .select()
                .apis(RequestHandlerSelectors.any())
                .paths(PathSelectors.regex("/api/.*"))
                .build();
    }
}

Upload controller:

@RestController
@RequestMapping(value = "/files")
public class FilesController {

    @Inject
    private Upload upload;

    @RequestMapping(method = RequestMethod.POST)
    public void addSource(@RequestParam MultipartFile file) throws IOException, InterruptedException {
        upload.process(file);
    }
}

Upload service:

@Component
public class Upload {

    @Async
    public void process(MultipartFile file) throws InterruptedException, IOException {
        sleep(2000);
        System.out.println(new String(IOUtils.readFully(file.getInputStream(), -1, false)));
    }
}

And now I get java.io.FileNotFoundException. I'm not sure what I'm missing here. Probably I'm doing something wrong as I couldn't find any error on this and I'd think this is very common use-case.

Do you make a post request or do you make an ajax request when you upload the file? — Periklis Douvitsas, Apr 15 '16 at 09:45
In addition, did you use the @Async annotation in service method or in controller? — Periklis Douvitsas, Apr 15 '16 at 10:24
It's a simple POST request (through swagger). I put @Async annotation on service method. — brgs, Apr 15 '16 at 10:45
Can you upload relevant code? It'll help us understand your attempt. — Manish Kothari, Apr 15 '16 at 11:07
Ok, I'll upload it as soon as I can (have to simplify first). — brgs, Apr 15 '16 at 11:08

score 55 · Accepted Answer · edited Oct 22 '19 at 08:09

55

You cannot hand over the MultipartFile paramter to your @Async Method in the way you do it.

When the addSource method ends, the MultipartFile runs out of scope and the resource is released. So the access inside your "process" method will fail. You build some kind of a race-condition this way. Springs DispatcherServlet uses StandardServletMultipartResolver.cleanupMultipart to cleanup those files. Put a breakpoint there to see when this method gets called upon retun of addSource(...).

You should do the following: Read the entire file into a Buffer inside the addSource method. Then pass the buffer to the process Method an let addSource return.

What you call "...processes the file..." is not "processing" the file, but reading it.

@Async
public void process(byte[] bs){
    System.out.println(new String(bs));
    //do some long running processing of bs here
}

@RequestMapping(method = RequestMethod.POST)
public void addSource(@RequestParam MultipartFile file) {
    upload.process(IOUtils.toByteArray(file));
}

edited Oct 22 '19 at 08:09

Martin Schröder

4,176
7
47
81

answered Apr 18 '16 at 09:35

TomB

1,010
10
14

Thank you! Btw I am doing some long processing here, it's just an example. – brgs Apr 20 '16 at 09:41
8

Nice explanation! Unfortunately, however, the suggested solution is a littlebit flawed: In enterprise environments, you often have files of sizes you cannot afford to buffer into memory like that. Sadly enough, `InputStreamResource` won't work either. Your best bet, as far as I could tell so far, is to copy contents to an own temp file (e.g. `File#createTempFile`) which you can clean up at the end of the processing thread. – Powerslave Mar 09 '18 at 14:45
2

Yes @Powerslave your are absolutely right. If you cannot afford buffering in memory you should use a filesystem backed buffer instead (aka a temp-file). I use the in memory approach in the example because it does not create new issues that arise with file handling (access restrictions, necessary cleanup task, etc.). – TomB Mar 12 '18 at 07:35
To add: the IOUtils is from Apache Commons IO (https://mvnrepository.com/artifact/commons-io/commons-io) – BigJ May 10 '20 at 11:21
Why not do `file.getBytes()` instead of `IOUtils.toByteArray(file)`. – web.learner Jul 07 '22 at 23:30

Spring async file upload and processing

1 Answers1

Linked