0

Spring boot shuts down without completing all the task.

I want to collect the data from the MYSQL database and then export this data as csv file.. However Spring boot shuts down after some time..

I tried following the example Here and Here but it seems like 

 CompletableFuture.allOf(tasks.toArray(new CompletableFuture[tasks.size()])).join();

doesnot wait untill all the tasks is finished.

NOTE: I have over 150 task for which i need to export data to the CSV file.

Code:

MainApplication.java

@SpringBootApplication
@EnableAsync
public class MainApplication implements CommandLineRunner {

@Autowired
private TaskRunner taskrunner;

public static void main(String[] args) throws Exception{
    SpringApplication application = new SpringApplication(MainApplication.class);
    application.setBannerMode(Banner.Mode.OFF);
    application.run(args);
  }

 public void run(String... args) throws Exception {
    taskrunner.executeTasks();
    exit(0);
   }
}

TaskRunner.java

@Service
public class TaskRunner {

@Autowired
public DataCollector dataCollector;

public void executeTask() throws Exception {

  final List<String> parameters = dataCollector.getParameters();
  List<CompletableFuture> tasks = new ArrayList<CompletableFuture>();

  for (String name : parameters) {
    try{
    tasks.add(CompletableFuture.runAsync(() -> dataCollector.ExportDataToCsv(name)));
    }catch(){
      ex.printStackTrace();
      System.out.println("Export failed for Param: "+name);
    }
  }

  CompletableFuture.allOf(tasks.toArray(new CompletableFuture[tasks.size()])).join();

  System.out.println("All Task Finished");
  }
}

DataCollector.java

public class DataCollector{

@Autowired
public DataRepository dataRepository;

 @Async("ThreadPoolTaskExecutor")
 public CompletableFuture<String> ExportDataToCsv(String tableName){
   // Code To export data to csv
 }
}

DataRepository.java

@Repository
public class DataRepository {

   @Qualifier("jdbcExportService")
   @Autowired
   public JdbcTemplate jdbcTemplate;

   public SqlRowSet getParamData(String param){
    String Statement = "select * FROM " + param;
    return jdbcTemplate.queryForRowSet(Statement);
  }
}

DatabaseConfiguration.java

 @Configuration
 public class DatabaseConfiguration {

@Bean(name="db")
@ConfigurationProperties(prefix = "spring.db")
public DataSource createExportDataSource(){
    return DataSourceBuilder.create().build();
}

@Bean(name = "jdbcExportService")
@Autowired
public JdbcTemplate createJdbcTemplateExportService(@Qualifier("db") DataSource exportServiceDS){
    return new JdbcTemplate(exportServiceDS);
 }
}

Output: 

2019-06-25 14:39:56.868  INFO 13163 --- [       Thread-2] o.s.s.concurrent.ThreadPoolTaskExecutor  : Shutting down ExecutorService 'ThreadPoolTaskExecutor'
2019-06-25 14:39:56.868  INFO 13163 --- [       Thread-2] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2019-06-25 14:39:56.997  INFO 13163 --- [       Thread-2] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2019-06-25 14:39:57.011 DEBUG 13163 --- [       Thread-2] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before shutdown stats (total=10, active=5, idle=5, waiting=0)
2019-06-25 14:39:57.371 DEBUG 13163 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.mysql.cj.jdbc.ConnectionImpl@2c373ded: (connection evicted)
2019-06-25 14:39:57.991 DEBUG 13163 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.mysql.cj.jdbc.ConnectionImpl@4856fa0b: (connection evicted)
    2019-06-25 14:39:57.994 DEBUG 13163 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.mysql.cj.jdbc.ConnectionImpl@375c83e: (connection evicted)
2019-06-25 14:39:57.995 DEBUG 13163 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.mysql.cj.jdbc.ConnectionImpl@417d7b5b: (connection evicted)
2019-06-25 14:39:57.996 DEBUG 13163 --- [nnection closer] com.zaxxer.hikari.pool.PoolBase          : HikariPool-1 - Closing connection com.mysql.cj.jdbc.ConnectionImpl@77ebb3e9: (connection evicted)
2019-06-25 14:39:58.094 DEBUG 13163 --- [       Thread-2] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After shutdown stats (total=0, active=0, idle=0, waiting=0)
2019-06-25 14:39:58.095  INFO 13163 --- [       Thread-2] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.
2019-06-25 14:39:58.651  WARN 13163 --- [ export-thread1] com.zaxxer.hikari.pool.ProxyConnection   : HikariPool-1 - Connection com.mysql.cj.jdbc.ConnectionImpl@4c18c2a2 marked as broken because of SQLSTATE(08003), ErrorCode(0)
Gabf Hann
  • 350
  • 4
  • 16
  • is there any reason as to why you are running a `@Async` task in CompletableFuture that is a wrapper for synchronous tasks? – Toerktumlare Jun 25 '19 at 12:47
  • you either use `@Async` https://stackoverflow.com/questions/29181057/how-to-check-that-async-call-completed-in-spring or you use `CompletableFuture` https://stackoverflow.com/questions/49839557/completablefuture-several-tasks Id prefer the `@Async` approach only, because of the threadpooling – Toerktumlare Jun 25 '19 at 12:50
  • Because i am trying to achieve async + multithreading... i have over 150 task which i want to run on 10 threads asynchronously.. therefore in my class Datacollector i use Thread (annotatuon) – Gabf Hann Jun 25 '19 at 22:24
  • Did you consider using Spring Batch to accomplish this? – Mohamed Makkaoui Jun 25 '19 at 22:33

1 Answers1

0

my guess after reading up a bit and i'm in no way a pro at this. So this could very well be wrong.

CompletableFuture#join waits until you can extract the resulting value from that CompletableFuture. CompletableFuture#get throws a checked exception and is interruptible while CompletableFuture#join is non-interruptible. But both do the same thing, extract the value from the completable future, and block if needed until they do.

What i think you are looking for since you want to print something when they are done is to use the CompletableFuture#thenAccept that will perform "something" when all are finished.

CompletableFuture<Void> allFutures = CompletableFuture.allOf(tasks.toArray(new CompletableFuture[tasks.size()]));
allFutures.thenAccept(Void -> System.out.println("All Task Finished"));

My guess is that it sets up all async tasks and then passes it all and then exits the application.

references:

java-8-completablefuture-in-action

completablefuture-join-vs-get

Toerktumlare
  • 12,548
  • 3
  • 35
  • 54