Monitor currently running process via spring batch process and notify if the throttle limit is reached

Question

I have a spring boot web application and I want to upload details from CSV files through a batch process. Files can be upload from any location and I want to restrict the process 3 jobs at a time. if already in process of 3 files, we have to give that information to UI as, "Throttle limit is being reached. Try after some time". How Could I achieve this? My current flow.xml is with master slave approach

    <!-- partitioner job -->
    <job id="partitionJob" xmlns="http://www.springframework.org/schema/batch">

    <!-- master step -->
            <step id="masterStep">
              <partition step="slave" partitioner="partitioner">
                    <handler grid-size="1" task-executor="taskExecutor" />
               </partition>
            </step>

    </job>

    <!-- each thread will run this job, with different stepExecutionContext
    values. -->
    <step id="slave" xmlns="http://www.springframework.org/schema/batch">
            <tasklet transaction-manager="transactionManager"  throttle-limit="3">
                <chunk reader="itemReader" processor="userItemProcessor" writer="itemWriter"
               commit-interval="10" />
                 <listeners>
                        <listener ref="stepJobListener" />
                  </listeners>
            </tasklet>
    </step>

Answer 1

The throttle-limit="3" that you've set on the step is for the threads that will run your step. For your use case, you should be looking for a way to configure the TaskExecutor that is used by the JobLauncher to limit the number of concurrent jobs.

You can for example configure the JobLauncher with a ThreadPoolTaskExecutor on which you set the queueCapacity to 3. With this configuration, if you submit more than 3 jobs at a time, the 4th submitted job will fail and you can show the error in your UI.

For the monitoring part, you can inspect the executor's queue size as described in Spring framework monitoring ThreadPoolTaskExecutor queue size