0

I have configured a spring batch tasklet to validate a xml file which is of size 3 gb.With just tasklet it is taking one minute to validate one xml file.Now i have configured local partitioning to validate two xml files in parallel

   <batch:job id="ValidationJob" incrementer="runIdIncrementer">
        <batch:step id="validationMaster" allow-start-if-complete="false">
                <batch:partition step="validationSlave"partitioner="filePartioner">
                    <batch:handler task-executor="taskExecutor" />
                </batch:partition>
            </batch:step>
    </batch:job>

<batch:step id="validationSlave">
        <batch:tasklet ref="XMLValidation">
        </batch:tasklet>
    </batch:step>

    <bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
            <property name="corePoolSize" value="2" />
            <property name="maxPoolSize" value="10" />
            <property name="allowCoreThreadTimeOut" value="false" />
    </bean>

My partitioner is simple it just create partitions to create partition for number of files in a directory.When i try to run two xml files in parallel it is taking more than 5minutes.Which ideally should be done in less than 2minutes.Where am i doing wrong

nanpakal
  • 971
  • 3
  • 18
  • 37
  • i think its not a spring batch problem, did you look into the problem of validating large xml files? like http://stackoverflow.com/questions/40663/validating-a-huge-xml-file – Michael Pralow Feb 16 '16 at 08:31
  • its not only the problem with validation.even storing records into database also behaving the same way. – nanpakal Feb 16 '16 at 08:42

0 Answers0